Jul 06 18:19:24.665184 ip-10-0-141-229 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 18:19:25.011144 ip-10-0-141-229 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 18:19:25.011144 ip-10-0-141-229 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 18:19:25.011144 ip-10-0-141-229 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 18:19:25.011144 ip-10-0-141-229 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 18:19:25.011144 ip-10-0-141-229 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 18:19:25.011144 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:24.999007    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 18:19:25.011144 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004004    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:19:25.011144 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004019    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:19:25.011144 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004024    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:19:25.011144 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004031    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:19:25.011144 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004037    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:19:25.011144 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004041    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:19:25.011144 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004046    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:19:25.011144 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004049    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004053    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004057    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004062    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004066    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004069    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004073    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004077    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004082    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004086    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004089    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004094    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004097    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004101    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004105    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004109    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004113    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004116    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004131    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004135    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004139    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:19:25.012211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004143    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004146    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004150    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004156    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004161    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004166    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004171    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004175    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004179    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004189    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004195    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004200    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004204    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004208    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004212    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004216    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004220    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004224    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:19:25.012846 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004227    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004231    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004235    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004239    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004243    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004247    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004251    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004254    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004260    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004264    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004267    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004271    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004274    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004278    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004281    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004285    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004289    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004293    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004296    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:19:25.013488 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004300    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004304    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004308    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004311    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004315    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004319    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004324    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004328    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004332    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004335    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004339    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004343    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004346    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004351    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004355    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004359    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004362    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004366    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004370    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004374    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:19:25.014128 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004378    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004382    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004386    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004390    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004393    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004397    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004400    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004405    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004409    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004413    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004416    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004420    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004424    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.004428    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005470    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005479    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005483    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005487    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005490    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005495    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:19:25.014761 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005499    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005502    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005509    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005515    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005519    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005523    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005528    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005533    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005537    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005542    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005546    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005551    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005555    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005559    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005563    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005567    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005572    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005575    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005579    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:19:25.015589 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005583    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005586    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005590    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005594    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005597    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005601    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005604    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005608    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005612    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005615    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005619    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005623    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005627    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005631    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005635    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005639    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005642    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005649    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005654    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:19:25.016211 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005658    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005662    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005666    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005670    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005675    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005678    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005682    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005686    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005689    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005693    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005696    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005700    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005704    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005707    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005711    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005715    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005718    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005723    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005727    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005731    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:19:25.016836 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005735    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005739    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005743    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005747    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005750    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005754    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005758    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005765    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005769    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005773    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005777    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005781    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005784    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005788    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005793    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005813    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005818    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005822    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005826    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:19:25.017482 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005830    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005834    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005838    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005842    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005846    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005850    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005853    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005857    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005861    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005865    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005868    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005872    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005876    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005879    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005883    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.005887    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006456    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006463    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006469    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:19:25.018110 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006474    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006479    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006483    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006486    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006490    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006494    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006498    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006502    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006506    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006512    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006515    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006519    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006523    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006526    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006530    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006534    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006539    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006543    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006547    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006551    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:19:25.018750 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006555    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006558    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006562    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006566    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006569    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006573    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006576    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006580    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006584    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006588    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006593    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006597    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006601    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006604    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006611    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006616    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006620    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006625    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006628    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:19:25.019406 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006632    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006635    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006639    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006644    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006648    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006652    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006656    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006660    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006664    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006667    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006671    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006675    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006679    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006683    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006686    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006690    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006693    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006704    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006708    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006712    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:19:25.020272 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006716    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006720    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006783    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006790    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006794    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006799    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006803    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006806    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006811    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006818    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006823    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006827    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006830    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006834    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006839    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006843    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006847    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006851    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006856    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:19:25.020944 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006860    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006864    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006868    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006871    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006875    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006882    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006886    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006890    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006893    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006898    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006916    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006921    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006925    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006929    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006933    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006937    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006941    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.006945    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007033    2581 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007051    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007062    2581 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 18:19:25.021558 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007069    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007082    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007087    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007093    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007098    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007103    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007108    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007113    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007118    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007123    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007128    2581 flags.go:64] FLAG: --cgroup-root=""
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007132    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007139    2581 flags.go:64] FLAG: --client-ca-file=""
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007143    2581 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007148    2581 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007157    2581 flags.go:64] FLAG: --cluster-domain=""
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007164    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007169    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007174    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007179    2581 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007184    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007188    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007193    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 18:19:25.022444 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007197    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007202    2581 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007206    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007211    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007215    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007219    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007225    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007229    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007233    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007237    2581 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007242    2581 flags.go:64] FLAG: --enable-server="true"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007254    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007263    2581 flags.go:64] FLAG: --event-burst="100"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007268    2581 flags.go:64] FLAG: --event-qps="50"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007272    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007277    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007281    2581 flags.go:64] FLAG: --eviction-hard=""
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007287    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007291    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007296    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007300    2581 flags.go:64] FLAG: --eviction-soft=""
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007306    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007311    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007315    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007320    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 18:19:25.023290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007326    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007330    2581 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007334    2581 flags.go:64] FLAG: --feature-gates=""
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007339    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007344    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007348    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007352    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007357    2581 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007361    2581 flags.go:64] FLAG: --help="false"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007365    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-141-229.ec2.internal"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007370    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007375    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007379    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007384    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007389    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007394    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007398    2581 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007401    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007405    2581 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007417    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007422    2581 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007426    2581 flags.go:64] FLAG: --kube-reserved=""
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007431    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007435    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 18:19:25.024536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007439    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007443    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007447    2581 flags.go:64] FLAG: --lock-file=""
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007451    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007457    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007462    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007469    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007473    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007480    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007484    2581 flags.go:64] FLAG: --logging-format="text"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007488    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007493    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007497    2581 flags.go:64] FLAG: --manifest-url=""
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007501    2581 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007507    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007512    2581 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007518    2581 flags.go:64] FLAG: --max-pods="110"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007523    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007527    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007531    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007536    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007540    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007544    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007549    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007558    2581 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 18:19:25.026401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007563    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007567    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007572    2581 flags.go:64] FLAG: --pod-cidr=""
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007576    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007584    2581 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007588    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007593    2581 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007598    2581 flags.go:64] FLAG: --port="10250"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007602    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007607    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f53109cde1fd508f"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007611    2581 flags.go:64] FLAG: --qos-reserved=""
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007618    2581 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007622    2581 flags.go:64] FLAG: --register-node="true"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007627    2581 flags.go:64] FLAG: --register-with-taints=""
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007632    2581 flags.go:64] FLAG: --registry-burst="10"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007636    2581 flags.go:64] FLAG: --registry-qps="5"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007644    2581 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007648    2581 flags.go:64] FLAG: --reserved-memory=""
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007654    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007658    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007663    2581 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007667    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007671    2581 flags.go:64] FLAG: --runonce="false"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007676    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007680    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 18:19:25.027466 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007685    2581 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007689    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007693    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007698    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007702    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007707    2581 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007711    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007715    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007719    2581 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007724    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007728    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007733    2581 flags.go:64] FLAG: --system-cgroups=""
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007738    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007745    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007749    2581 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007754    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007762    2581 flags.go:64] FLAG: --tls-min-version=""
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007766    2581 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007772    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007777    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007781    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007786    2581 flags.go:64] FLAG: --v="2"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007791    2581 flags.go:64] FLAG: --version="false"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007798    2581 flags.go:64] FLAG: --vmodule=""
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007804    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 18:19:25.028251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.007809    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.007989    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.007996    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008000    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008005    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008008    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008012    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008016    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008019    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008023    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008029    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008033    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008038    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008043    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008047    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008052    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008056    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008060    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008064    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008068    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:19:25.028920 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008072    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008076    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008079    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008083    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008087    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008092    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008095    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008099    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008103    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008106    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008112    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008116    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008120    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008124    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008127    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008132    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008136    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008140    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008144    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008147    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:19:25.029521 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008151    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008155    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008159    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008163    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008166    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008169    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008174    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008178    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008181    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008185    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008189    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008192    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008196    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008200    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008203    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008207    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008211    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008216    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008220    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008223    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:19:25.030131 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008227    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008230    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008236    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008240    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008243    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008247    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008251    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008256    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008260    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008263    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008268    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008273    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008278    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008282    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008286    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008290    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008294    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008298    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008301    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008305    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:19:25.118328 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008309    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:19:25.079336 ip-10-0-141-229 systemd[1]: Started Kubernetes Kubelet.
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008313    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008316    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008320    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008325    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008330    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008334    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008339    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008343    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008348    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008353    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008356    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008360    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008364    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008369    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008372    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008376    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008380    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008384    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:19:25.119425 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.008388    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:19:25.120827 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.008395    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 18:19:25.120827 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.015779    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 18:19:25.120827 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.015799    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 18:19:25.120827 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.015820    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 18:19:25.120827 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.015829    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 18:19:25.120827 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015899    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:19:25.120827 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015920    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:19:25.120827 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015924    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:19:25.120827 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015927    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:19:25.120827 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015930    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:19:25.120827 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015934    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:19:25.120827 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015937    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:19:25.120827 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015940    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:19:25.120827 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015943    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015946    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015949    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015952    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015955    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015958    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015960    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015963    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015966    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015969    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015972    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015974    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015977    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015979    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015982    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015984    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015987    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015990    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015992    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:19:25.121553 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015995    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.015997    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016007    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016010    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016013    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016016    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016019    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016024    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016027    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016029    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016032    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016034    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016038    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016042    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016045    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016048    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016051    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016054    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016056    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016059    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:19:25.122613 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016063    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016066    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016068    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016072    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016075    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016078    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016080    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016083    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016086    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016089    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016092    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016094    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016097    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016099    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016102    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016110    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016113    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016115    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016118    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016120    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:19:25.128343 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016123    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016126    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016129    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016131    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016134    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016136    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016139    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016141    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016144    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016146    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016149    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016152    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016155    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016158    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016161    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016163    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016166    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016169    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016171    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016174    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:19:25.129039 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016176    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:19:25.129752 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016179    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:19:25.129752 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016182    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:19:25.129752 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016184    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:19:25.129752 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016187    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:19:25.129752 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016189    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:19:25.129752 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016192    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:19:25.129752 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016194    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:19:25.129752 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016203    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:19:25.129752 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016206    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:19:25.129752 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016209    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:19:25.129752 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016212    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:19:25.129752 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.016218    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 18:19:25.129752 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016377    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:19:25.129752 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016382    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016385    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016388    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016391    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016394    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016397    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016399    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016403    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016407    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016410    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016413    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016416    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016419    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016422    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016424    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016427    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016429    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016432    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016435    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:19:25.131560 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016437    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016440    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016442    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016445    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016448    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016450    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016452    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016455    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016463    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016466    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016468    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016471    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016473    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016476    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016478    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016481    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016483    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016486    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016488    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016491    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:19:25.132267 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016493    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016496    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016499    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016501    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016504    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016506    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016509    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016511    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016514    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016516    2581 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016519    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016521    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016524    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016527    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016529    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016532    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016535    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016537    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016544    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016547    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:19:25.133007 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016549    2581 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016558    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016561    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016563    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016566    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016568    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016571    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016574    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016577    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016581    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016583    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016586    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016589    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016591    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016593    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016596    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016600    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016605    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016608    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:19:25.135508 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016611    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016614    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016616    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016619    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016621    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016624    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016627    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016629    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016632    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016634    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016637    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016641    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016644    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016646    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016649    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016658    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016660    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016663    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016666    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:19:25.136222 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:25.016668    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:19:25.138030 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.016673    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 18:19:25.138030 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.016829    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 18:19:25.138030 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.020782    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 18:19:25.138030 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.022078    2581 server.go:1019] "Starting client certificate rotation"
Jul 06 18:19:25.138030 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.022198    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 18:19:25.138030 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.022246    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 18:19:25.138030 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.026061    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 18:19:25.138030 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.029712    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 18:19:25.138030 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.045230    2581 log.go:25] "Validated CRI v1 runtime API"
Jul 06 18:19:25.138030 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.047453    2581 log.go:25] "Validated CRI v1 image API"
Jul 06 18:19:25.138030 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.048265    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 18:19:25.138030 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.050978    2581 fs.go:135] Filesystem UUIDs: map[77e42d1c-b5a7-4dfe-805c-f272ff5c91d3:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 efd9cfa3-e5d1-4405-99f2-93977d28883e:/dev/nvme0n1p3]
Jul 06 18:19:25.138737 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.050995    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 18:19:26.487627 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.054705    2581 manager.go:216] Machine: {Timestamp:2026-07-06 18:19:25.054212723 +0000 UTC m=+0.284492188 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098766 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21231985006bb814e3041c202f7786 SystemUUID:ec212319-8500-6bb8-14e3-041c202f7786 BootID:fa9774c9-aba2-4d73-bf83-2adbe5f9f2a8 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ca:c1:97:c9:6f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ca:c1:97:c9:6f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:36:9c:75:4b:fb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 18:19:26.487627 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.054805    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 18:19:26.487627 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.054928    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 18:19:26.487627 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.057566    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 18:19:26.487627 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.057593    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-229.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 18:19:26.487627 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.058273    2581 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 18:19:26.487627 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.058283    2581 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 18:19:26.487627 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.058291    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.058914    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.060253    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.063625    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.063739    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.064480    2581 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.064491    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.064501    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.064518    2581 kubelet.go:398] "Adding apiserver pod source"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.064534    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.064829    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.064844    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.070960    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.073043    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.073075    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.074128    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.074145    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.074160    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.074166    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.074175    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.074184    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.074193    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.074202    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.074210    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.074215    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.074228    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.074233    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.074819    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.074826    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.078524    2581 server.go:1268] "Started kubelet"
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.079003    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 18:19:26.488403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:25.080738    2581 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 18:19:27.335492 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.335420    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 18:19:27.336055 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.335583    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 18:19:27.339157 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.339135    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 18:19:27.340207 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.340090    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 18:19:27.340207 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.340109    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 18:19:27.340418 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.340396    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found"
Jul 06 18:19:27.340855 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.340835    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 18:19:27.340855 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.340858    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 18:19:27.340993 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.340841    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 18:19:27.341068 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.341052    2581 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 18:19:27.341135 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.341069    2581 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 18:19:27.341824 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.341802    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-229.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 18:19:27.341915 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.341833    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-229.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 18:19:27.341915 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.341865    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 18:19:27.342221 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.342206    2581 factory.go:55] Registering systemd factory
Jul 06 18:19:27.342317 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.342289    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 18:19:27.342317 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.342307    2581 factory.go:223] Registration of the systemd container factory successfully
Jul 06 18:19:27.342555 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.342540    2581 factory.go:153] Registering CRI-O factory
Jul 06 18:19:27.342596 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.342561    2581 factory.go:223] Registration of the crio container factory successfully
Jul 06 18:19:27.342635 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.342610    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 18:19:27.342667 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.342639    2581 factory.go:103] Registering Raw factory
Jul 06 18:19:27.342667 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.342654    2581 manager.go:1196] Started watching for new ooms in manager
Jul 06 18:19:27.343124 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.343093    2581 manager.go:349] Starting recovery of all containers
Jul 06 18:19:27.349627 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.349589    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 18:19:27.353724 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.352639    2581 manager.go:354] Recovery completed
Jul 06 18:19:27.353724 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.352954    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-229.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 18:19:27.353724 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.353046    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 18:19:27.356997 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.356973    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 18:19:27.357376 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.357364    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:19:27.359027 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.359001    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:19:27.359103 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.359045    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:19:27.359103 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.359060    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:19:27.359698 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.359682    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 18:19:27.359698 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.359697    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 18:19:27.359814 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.359716    2581 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 18:19:27.359814 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.359720    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:19:27.360607 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.360589    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:19:27.360665 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.360618    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:19:27.360665 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.360630    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:19:27.361010 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.360999    2581 policy_none.go:49] "None policy: Start"
Jul 06 18:19:27.361060 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.361015    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 18:19:27.361060 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.361026    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 18:19:27.361933 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.361920    2581 policy_none.go:47] "Start"
Jul 06 18:19:27.363189 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.362348    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-229.ec2.internal.18bfc64232724745  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-229.ec2.internal,UID:ip-10-0-141-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-229.ec2.internal,},FirstTimestamp:2026-07-06 18:19:25.078497093 +0000 UTC m=+0.308776549,LastTimestamp:2026-07-06 18:19:25.078497093 +0000 UTC m=+0.308776549,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-229.ec2.internal,}"
Jul 06 18:19:27.373494 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.373426    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-229.ec2.internal.18bfc642ba606a09  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-229.ec2.internal,UID:ip-10-0-141-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-229.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-229.ec2.internal,},FirstTimestamp:2026-07-06 18:19:27.359027721 +0000 UTC m=+1.332477226,LastTimestamp:2026-07-06 18:19:27.359027721 +0000 UTC m=+1.332477226,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-229.ec2.internal,}"
Jul 06 18:19:27.379152 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.379130    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qvdqd"
Jul 06 18:19:27.382954 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.382685    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-229.ec2.internal.18bfc642ba60c8ac  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-229.ec2.internal,UID:ip-10-0-141-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-229.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-229.ec2.internal,},FirstTimestamp:2026-07-06 18:19:27.359051948 +0000 UTC m=+1.332501453,LastTimestamp:2026-07-06 18:19:27.359051948 +0000 UTC m=+1.332501453,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-229.ec2.internal,}"
Jul 06 18:19:27.387177 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.387160    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qvdqd"
Jul 06 18:19:27.395415 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.395401    2581 manager.go:337] "Starting Device Plugin manager"
Jul 06 18:19:27.395494 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.395437    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 18:19:27.395494 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.395449    2581 server.go:87] "Starting device plugin registration server"
Jul 06 18:19:27.395752 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.395719    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 18:19:27.395833 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.395775    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 18:19:27.395883 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.395868    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 18:19:27.395981 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.395966    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 18:19:27.395981 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.395979    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 18:19:27.396488 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.396466    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 18:19:27.396558 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.396522    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-229.ec2.internal\" not found"
Jul 06 18:19:27.465155 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.465124    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 18:19:27.467229 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.465167    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 18:19:27.467229 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.465191    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 18:19:27.467229 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.465223    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 18:19:27.468026 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.468002    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:19:27.496052 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.496028    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:19:27.496982 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.496959    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:19:27.497080 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.496989    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:19:27.497080 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.497001    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:19:27.497080 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.497029    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.506018 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.505999    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.506089 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.506022    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-229.ec2.internal\": node \"ip-10-0-141-229.ec2.internal\" not found"
Jul 06 18:19:27.553039 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.553007    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found"
Jul 06 18:19:27.565333 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.565308    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal"]
Jul 06 18:19:27.565408 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.565369    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:19:27.566866 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.566850    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:19:27.566952 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.566876    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:19:27.566952 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.566886    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:19:27.569072 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.569059    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:19:27.569190 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.569168    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.569248 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.569203    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:19:27.569836 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.569810    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:19:27.569836 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.569818    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:19:27.569836 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.569837    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:19:27.569973 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.569837    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:19:27.569973 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.569852    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:19:27.569973 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.569867    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:19:27.572135 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.572119    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.572201 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.572152    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:19:27.572913 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.572887    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:19:27.573021 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.572923    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:19:27.573021 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.572935    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:19:27.599334 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.599316    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-229.ec2.internal\" not found" node="ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.603536 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.603511    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-229.ec2.internal\" not found" node="ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.642629 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.642606    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd320ee8d7c30c12d7b97d56aa7806c5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal\" (UID: \"bd320ee8d7c30c12d7b97d56aa7806c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.642778 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.642639    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd320ee8d7c30c12d7b97d56aa7806c5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal\" (UID: \"bd320ee8d7c30c12d7b97d56aa7806c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.642778 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.642676    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dd0c9e68fa1554fd01092c3bef64ba0f-config\") pod \"kube-apiserver-proxy-ip-10-0-141-229.ec2.internal\" (UID: \"dd0c9e68fa1554fd01092c3bef64ba0f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.653789 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.653769    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found"
Jul 06 18:19:27.743088 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.743010    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dd0c9e68fa1554fd01092c3bef64ba0f-config\") pod \"kube-apiserver-proxy-ip-10-0-141-229.ec2.internal\" (UID: \"dd0c9e68fa1554fd01092c3bef64ba0f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.743088 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.743047    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd320ee8d7c30c12d7b97d56aa7806c5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal\" (UID: \"bd320ee8d7c30c12d7b97d56aa7806c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.743233 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.743070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd320ee8d7c30c12d7b97d56aa7806c5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal\" (UID: \"bd320ee8d7c30c12d7b97d56aa7806c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.743233 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.743120    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dd0c9e68fa1554fd01092c3bef64ba0f-config\") pod \"kube-apiserver-proxy-ip-10-0-141-229.ec2.internal\" (UID: \"dd0c9e68fa1554fd01092c3bef64ba0f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.743233 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.743128    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd320ee8d7c30c12d7b97d56aa7806c5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal\" (UID: \"bd320ee8d7c30c12d7b97d56aa7806c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.743233 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.743139    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd320ee8d7c30c12d7b97d56aa7806c5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal\" (UID: \"bd320ee8d7c30c12d7b97d56aa7806c5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.754155 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.754118    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found"
Jul 06 18:19:27.855043 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.855013    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found"
Jul 06 18:19:27.902221 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.902184    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.906050 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:27.906032    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal"
Jul 06 18:19:27.955984 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:27.955947    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found"
Jul 06 18:19:28.056501 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:28.056419    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found"
Jul 06 18:19:28.157094 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:28.157064    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found"
Jul 06 18:19:28.257731 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:28.257702    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found"
Jul 06 18:19:28.278908 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.278868    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 18:19:28.279054 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.279037    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 18:19:28.358285 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:28.358214    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found"
Jul 06 18:19:28.390390 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.390332    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 18:14:27 +0000 UTC" deadline="2027-12-18 10:40:43.986891476 +0000 UTC"
Jul 06 18:19:28.390390 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.390386    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12712h21m15.596508519s"
Jul 06 18:19:28.459068 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:28.459029    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found"
Jul 06 18:19:28.556653 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.556619    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:19:28.558034 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.558017    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:19:28.558107 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.558049    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:19:28.558107 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.558059    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:19:28.559240 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:28.559222    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found"
Jul 06 18:19:28.624310 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.624251    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:19:28.659592 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:28.659558    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-229.ec2.internal\" not found"
Jul 06 18:19:28.665808 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.665789    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:19:28.698901 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:28.698871    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd320ee8d7c30c12d7b97d56aa7806c5.slice/crio-0ca75e5c14c2784b41fbca164f2ec38e2ac2618697ef34603937b51a43ea620f WatchSource:0}: Error finding container 0ca75e5c14c2784b41fbca164f2ec38e2ac2618697ef34603937b51a43ea620f: Status 404 returned error can't find the container with id 0ca75e5c14c2784b41fbca164f2ec38e2ac2618697ef34603937b51a43ea620f
Jul 06 18:19:28.699148 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:28.699128    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd0c9e68fa1554fd01092c3bef64ba0f.slice/crio-548260d878a8b787b11285ce934b09bd006d2eab3603bce2112d96b8c06dbc45 WatchSource:0}: Error finding container 548260d878a8b787b11285ce934b09bd006d2eab3603bce2112d96b8c06dbc45: Status 404 returned error can't find the container with id 548260d878a8b787b11285ce934b09bd006d2eab3603bce2112d96b8c06dbc45
Jul 06 18:19:28.705133 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.705118    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 18:19:28.748116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.748092    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:19:28.840781 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.840722    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal"
Jul 06 18:19:28.853428 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.853403    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 18:19:28.854234 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.854217    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal"
Jul 06 18:19:28.862854 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:28.862836    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 18:19:29.336169 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.336134    2581 apiserver.go:52] "Watching apiserver"
Jul 06 18:19:29.341579 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.341482    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 18:19:29.344332 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.344307    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 18:19:29.345376 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.345346    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-wjgsb","openshift-network-diagnostics/network-check-target-rx4jm","openshift-network-operator/iptables-alerter-cvntg","openshift-ovn-kubernetes/ovnkube-node-nv4mk","kube-system/global-pull-secret-syncer-zdtg9","kube-system/konnectivity-agent-rqp4s","kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf","openshift-cluster-node-tuning-operator/tuned-7bxlq","openshift-dns/node-resolver-mr54p","openshift-image-registry/node-ca-s2ktj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal","openshift-multus/multus-additional-cni-plugins-6whc5","openshift-multus/multus-h4c6h"]
Jul 06 18:19:29.347933 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.347910    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mr54p"
Jul 06 18:19:29.350354 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.350319    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.350718 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.350696    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 18:19:29.350877 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.350861    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dc9vb\""
Jul 06 18:19:29.351059 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.351040    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 18:19:29.352227 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.352208    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cvntg"
Jul 06 18:19:29.352703 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.352679    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 18:19:29.352816 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.352780    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 18:19:29.353283 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.353113    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 18:19:29.353630 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.353474    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 18:19:29.353630 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.353554    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 18:19:29.354169 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.353976    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 18:19:29.354169 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.354026    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vbtr4\""
Jul 06 18:19:29.354998 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.354541    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.354998 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.354813    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 18:19:29.355141 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.355066    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 18:19:29.355447 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.355429    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 18:19:29.355656 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.355630    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n9jnl\""
Jul 06 18:19:29.356893 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.356842    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 18:19:29.357153 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.357129    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 18:19:29.357244 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.357159    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:29.357300 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:29.357255    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:29.357801 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.357600    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 18:19:29.357801 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.357609    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 18:19:29.358527 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.358372    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2r8cz\""
Jul 06 18:19:29.359119 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.359100    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 18:19:29.359219 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.359122    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 18:19:29.359483 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.359462    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rqp4s"
Jul 06 18:19:29.362188 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.362030    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8trdc\""
Jul 06 18:19:29.362188 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.362100    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 18:19:29.362188 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.362109    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 18:19:29.362379 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.362343    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.364712 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.364518    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.364712 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.364686    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-v7rjh\""
Jul 06 18:19:29.365473 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.365271    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 18:19:29.368269 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.368250    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 18:19:29.368360 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.368270    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 18:19:29.368723 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.368704    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:29.368833 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:29.368802    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:29.369516 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.369488    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jvj2d\""
Jul 06 18:19:29.373507 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.373443    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:29.373602 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:29.373514    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:29.375638 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.375617    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s2ktj"
Jul 06 18:19:29.377922 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.377899    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 18:19:29.378025 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.378005    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.378247 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.378222    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cx6j6\""
Jul 06 18:19:29.378526 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.378380    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 18:19:29.378526 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.378457    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 18:19:29.380352 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.380332    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-krr5r\""
Jul 06 18:19:29.380508 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.380490    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 18:19:29.380576 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.380515    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 18:19:29.380828 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.380808    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 18:19:29.399328 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.399301    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2kp72"
Jul 06 18:19:29.407278 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.407236    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2kp72"
Jul 06 18:19:29.441934 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.441905    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 18:19:29.452168 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.452140    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-cni-netd\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.452324 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.452174    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-var-lib-cni-bin\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.452324 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.452193    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr7pr\" (UniqueName: \"kubernetes.io/projected/68c5ca38-8b7c-437c-a608-dc779f7b3628-kube-api-access-kr7pr\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.452324 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.452220    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-sys-fs\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.452777 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.452760    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xnbw\" (UniqueName: \"kubernetes.io/projected/53792e6a-1284-4ac6-8370-bb836200eb67-kube-api-access-5xnbw\") pod \"node-resolver-mr54p\" (UID: \"53792e6a-1284-4ac6-8370-bb836200eb67\") " pod="openshift-dns/node-resolver-mr54p"
Jul 06 18:19:29.452866 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.452791    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c840b50-b644-436f-bbb9-bcaac772988a-cnibin\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.452866 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.452811    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c840b50-b644-436f-bbb9-bcaac772988a-cni-binary-copy\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.452866 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.452845    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/34b35881-3a46-4ae5-8e1b-0880314612ec-ovnkube-config\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.453010 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.452886    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-var-lib-cni-multus\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.453010 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.452909    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0c840b50-b644-436f-bbb9-bcaac772988a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.453010 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.452928    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-modprobe-d\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.453010 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.452952    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-kubernetes\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.453010 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.452988    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-node-log\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.453212 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453021    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-log-socket\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.453212 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5rbq\" (UniqueName: \"kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq\") pod \"network-check-target-rx4jm\" (UID: \"bfe5b327-4a68-43d2-90e5-ebe6e2670be0\") " pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:29.453212 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453088    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-socket-dir\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.453212 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453123    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-device-dir\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.453212 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453157    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-sysctl-d\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.453212 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453184    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-tuned\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.453212 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453212    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-slash\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.453541 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453237    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.453541 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0c840b50-b644-436f-bbb9-bcaac772988a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.453541 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453276    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3f857b5c-ed1c-42a0-8960-524770d6f07e-kubelet-config\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:29.453541 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453298    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-systemd\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.453541 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453335    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-host\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.453541 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453364    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d80ee81b-bb99-4520-a431-3b54ec3d9134-host-slash\") pod \"iptables-alerter-cvntg\" (UID: \"d80ee81b-bb99-4520-a431-3b54ec3d9134\") " pod="openshift-network-operator/iptables-alerter-cvntg"
Jul 06 18:19:29.453541 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453384    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-var-lib-kubelet\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.453541 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453401    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-system-cni-dir\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.453541 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453433    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68c5ca38-8b7c-437c-a608-dc779f7b3628-cni-binary-copy\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.453541 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453474    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-var-lib-kubelet\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.453541 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453500    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-run-multus-certs\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.453541 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453532    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53792e6a-1284-4ac6-8370-bb836200eb67-tmp-dir\") pod \"node-resolver-mr54p\" (UID: \"53792e6a-1284-4ac6-8370-bb836200eb67\") " pod="openshift-dns/node-resolver-mr54p"
Jul 06 18:19:29.454011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453555    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9d4a8d7b-f63e-46cc-b99b-fe9b655c3388-agent-certs\") pod \"konnectivity-agent-rqp4s\" (UID: \"9d4a8d7b-f63e-46cc-b99b-fe9b655c3388\") " pod="kube-system/konnectivity-agent-rqp4s"
Jul 06 18:19:29.454011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-var-lib-openvswitch\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.454011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453606    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtsxk\" (UniqueName: \"kubernetes.io/projected/d80ee81b-bb99-4520-a431-3b54ec3d9134-kube-api-access-dtsxk\") pod \"iptables-alerter-cvntg\" (UID: \"d80ee81b-bb99-4520-a431-3b54ec3d9134\") " pod="openshift-network-operator/iptables-alerter-cvntg"
Jul 06 18:19:29.454011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453621    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3f857b5c-ed1c-42a0-8960-524770d6f07e-dbus\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:29.454011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453645    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-sysconfig\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.454011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453678    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-kubelet\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.454011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453707    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-run-ovn\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.454011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453729    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-run-k8s-cni-cncf-io\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.454011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453775    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-hostroot\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.454011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453797    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhq7x\" (UniqueName: \"kubernetes.io/projected/02f96065-7946-46c4-ad55-a5a5a0fc6520-kube-api-access-rhq7x\") pod \"node-ca-s2ktj\" (UID: \"02f96065-7946-46c4-ad55-a5a5a0fc6520\") " pod="openshift-image-registry/node-ca-s2ktj"
Jul 06 18:19:29.454011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453814    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-run-ovn-kubernetes\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.454011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-multus-conf-dir\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.454011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c840b50-b644-436f-bbb9-bcaac772988a-os-release\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.454011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453882    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:29.454550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453921    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02f96065-7946-46c4-ad55-a5a5a0fc6520-host\") pod \"node-ca-s2ktj\" (UID: \"02f96065-7946-46c4-ad55-a5a5a0fc6520\") " pod="openshift-image-registry/node-ca-s2ktj"
Jul 06 18:19:29.454550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453954    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/02f96065-7946-46c4-ad55-a5a5a0fc6520-serviceca\") pod \"node-ca-s2ktj\" (UID: \"02f96065-7946-46c4-ad55-a5a5a0fc6520\") " pod="openshift-image-registry/node-ca-s2ktj"
Jul 06 18:19:29.454550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-run-netns\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.454550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.453986    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d80ee81b-bb99-4520-a431-3b54ec3d9134-iptables-alerter-script\") pod \"iptables-alerter-cvntg\" (UID: \"d80ee81b-bb99-4520-a431-3b54ec3d9134\") " pod="openshift-network-operator/iptables-alerter-cvntg"
Jul 06 18:19:29.454550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454001    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-run\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.454550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454025    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs\") pod \"network-metrics-daemon-wjgsb\" (UID: \"a331ebbc-53c2-44a9-ab94-9c308b480dfa\") " pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:29.454550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454044    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/34b35881-3a46-4ae5-8e1b-0880314612ec-ovnkube-script-lib\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.454550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454065    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-os-release\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.454550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454092    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6dq4\" (UniqueName: \"kubernetes.io/projected/12363263-5b1e-45b9-9ea0-c485ca8766f4-kube-api-access-k6dq4\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.454550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454119    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-sys\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.454550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454166    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-lib-modules\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.454550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454232    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4c4c\" (UniqueName: \"kubernetes.io/projected/a331ebbc-53c2-44a9-ab94-9c308b480dfa-kube-api-access-l4c4c\") pod \"network-metrics-daemon-wjgsb\" (UID: \"a331ebbc-53c2-44a9-ab94-9c308b480dfa\") " pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:29.454550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454278    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-run-netns\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.454550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454304    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-etc-openvswitch\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.455176 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454328    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-run-openvswitch\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.455176 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454362    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-registration-dir\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.455176 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454409    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/34b35881-3a46-4ae5-8e1b-0880314612ec-env-overrides\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.455176 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454432    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcd2c\" (UniqueName: \"kubernetes.io/projected/34b35881-3a46-4ae5-8e1b-0880314612ec-kube-api-access-jcd2c\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.455176 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454472    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-multus-cni-dir\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.455176 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-cnibin\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.455176 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-multus-socket-dir-parent\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.455176 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454562    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/53792e6a-1284-4ac6-8370-bb836200eb67-hosts-file\") pod \"node-resolver-mr54p\" (UID: \"53792e6a-1284-4ac6-8370-bb836200eb67\") " pod="openshift-dns/node-resolver-mr54p"
Jul 06 18:19:29.455176 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454599    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c840b50-b644-436f-bbb9-bcaac772988a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.455176 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454632    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2744r\" (UniqueName: \"kubernetes.io/projected/0c840b50-b644-436f-bbb9-bcaac772988a-kube-api-access-2744r\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.455176 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454656    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-run-systemd\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.455176 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454679    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/34b35881-3a46-4ae5-8e1b-0880314612ec-ovn-node-metrics-cert\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.455176 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454703    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-etc-selinux\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.455652 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454726    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c840b50-b644-436f-bbb9-bcaac772988a-system-cni-dir\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.455652 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9d4a8d7b-f63e-46cc-b99b-fe9b655c3388-konnectivity-ca\") pod \"konnectivity-agent-rqp4s\" (UID: \"9d4a8d7b-f63e-46cc-b99b-fe9b655c3388\") " pod="kube-system/konnectivity-agent-rqp4s"
Jul 06 18:19:29.455652 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454813    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/560020e3-64da-4cfa-8cd8-178b1bd54a55-tmp\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.455652 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454843    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tww6h\" (UniqueName: \"kubernetes.io/projected/560020e3-64da-4cfa-8cd8-178b1bd54a55-kube-api-access-tww6h\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.455652 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454868    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-systemd-units\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.455652 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454889    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-etc-kubernetes\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.455652 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454913    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.455652 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454935    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-sysctl-conf\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.455652 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454957    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/68c5ca38-8b7c-437c-a608-dc779f7b3628-multus-daemon-config\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.455652 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.454976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-cni-bin\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.469928 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.469876    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal" event={"ID":"dd0c9e68fa1554fd01092c3bef64ba0f","Type":"ContainerStarted","Data":"548260d878a8b787b11285ce934b09bd006d2eab3603bce2112d96b8c06dbc45"}
Jul 06 18:19:29.471187 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.471156    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" event={"ID":"bd320ee8d7c30c12d7b97d56aa7806c5","Type":"ContainerStarted","Data":"0ca75e5c14c2784b41fbca164f2ec38e2ac2618697ef34603937b51a43ea620f"}
Jul 06 18:19:29.555539 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555491    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-hostroot\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.555897 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555541    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhq7x\" (UniqueName: \"kubernetes.io/projected/02f96065-7946-46c4-ad55-a5a5a0fc6520-kube-api-access-rhq7x\") pod \"node-ca-s2ktj\" (UID: \"02f96065-7946-46c4-ad55-a5a5a0fc6520\") " pod="openshift-image-registry/node-ca-s2ktj"
Jul 06 18:19:29.555897 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-run-ovn-kubernetes\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.555897 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555594    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-multus-conf-dir\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.555897 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555614    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c840b50-b644-436f-bbb9-bcaac772988a-os-release\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.555897 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:29.555897 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555657    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02f96065-7946-46c4-ad55-a5a5a0fc6520-host\") pod \"node-ca-s2ktj\" (UID: \"02f96065-7946-46c4-ad55-a5a5a0fc6520\") " pod="openshift-image-registry/node-ca-s2ktj"
Jul 06 18:19:29.555897 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/02f96065-7946-46c4-ad55-a5a5a0fc6520-serviceca\") pod \"node-ca-s2ktj\" (UID: \"02f96065-7946-46c4-ad55-a5a5a0fc6520\") " pod="openshift-image-registry/node-ca-s2ktj"
Jul 06 18:19:29.555897 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555701    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-run-netns\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.555897 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d80ee81b-bb99-4520-a431-3b54ec3d9134-iptables-alerter-script\") pod \"iptables-alerter-cvntg\" (UID: \"d80ee81b-bb99-4520-a431-3b54ec3d9134\") " pod="openshift-network-operator/iptables-alerter-cvntg"
Jul 06 18:19:29.555897 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555763    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-run\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.555897 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555789    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs\") pod \"network-metrics-daemon-wjgsb\" (UID: \"a331ebbc-53c2-44a9-ab94-9c308b480dfa\") " pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:29.555897 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555811    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/34b35881-3a46-4ae5-8e1b-0880314612ec-ovnkube-script-lib\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.555897 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-os-release\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.555897 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6dq4\" (UniqueName: \"kubernetes.io/projected/12363263-5b1e-45b9-9ea0-c485ca8766f4-kube-api-access-k6dq4\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.556660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555883    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-sys\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.556660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555906    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-lib-modules\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.556660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555930    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4c4c\" (UniqueName: \"kubernetes.io/projected/a331ebbc-53c2-44a9-ab94-9c308b480dfa-kube-api-access-l4c4c\") pod \"network-metrics-daemon-wjgsb\" (UID: \"a331ebbc-53c2-44a9-ab94-9c308b480dfa\") " pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:29.556660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555953    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-run-netns\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.556660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.555978    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-etc-openvswitch\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.556660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556000    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-run-openvswitch\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.556660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556023    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-registration-dir\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.556660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556055    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/34b35881-3a46-4ae5-8e1b-0880314612ec-env-overrides\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.556660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556078    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcd2c\" (UniqueName: \"kubernetes.io/projected/34b35881-3a46-4ae5-8e1b-0880314612ec-kube-api-access-jcd2c\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.556660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556104    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-multus-cni-dir\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.556660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556135    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-cnibin\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.556660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556159    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-multus-socket-dir-parent\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.556660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556183    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/53792e6a-1284-4ac6-8370-bb836200eb67-hosts-file\") pod \"node-resolver-mr54p\" (UID: \"53792e6a-1284-4ac6-8370-bb836200eb67\") " pod="openshift-dns/node-resolver-mr54p"
Jul 06 18:19:29.556660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556208    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c840b50-b644-436f-bbb9-bcaac772988a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.557418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556249    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2744r\" (UniqueName: \"kubernetes.io/projected/0c840b50-b644-436f-bbb9-bcaac772988a-kube-api-access-2744r\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.557418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556339    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-etc-openvswitch\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.557418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556323    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-run-systemd\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.557418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556406    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/34b35881-3a46-4ae5-8e1b-0880314612ec-ovn-node-metrics-cert\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.557418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556411    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-run-systemd\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.557418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556435    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-etc-selinux\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.557418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556462    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-hostroot\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.557418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c840b50-b644-436f-bbb9-bcaac772988a-system-cni-dir\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.557418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556498    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c840b50-b644-436f-bbb9-bcaac772988a-system-cni-dir\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.557418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556498    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9d4a8d7b-f63e-46cc-b99b-fe9b655c3388-konnectivity-ca\") pod \"konnectivity-agent-rqp4s\" (UID: \"9d4a8d7b-f63e-46cc-b99b-fe9b655c3388\") " pod="kube-system/konnectivity-agent-rqp4s"
Jul 06 18:19:29.557418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556514    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-multus-cni-dir\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.557418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556537    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/560020e3-64da-4cfa-8cd8-178b1bd54a55-tmp\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.557418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556566    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tww6h\" (UniqueName: \"kubernetes.io/projected/560020e3-64da-4cfa-8cd8-178b1bd54a55-kube-api-access-tww6h\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.557418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556572    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-sys\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.557418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556595    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-systemd-units\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.558116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-etc-kubernetes\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.558116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556663    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.558116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556687    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-sysctl-conf\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.558116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556715    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/68c5ca38-8b7c-437c-a608-dc779f7b3628-multus-daemon-config\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.558116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-cni-bin\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.558116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556767    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c840b50-b644-436f-bbb9-bcaac772988a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.558116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556778    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-cni-netd\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.558116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556576    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-cnibin\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.558116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-var-lib-cni-bin\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.558116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556817    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/53792e6a-1284-4ac6-8370-bb836200eb67-hosts-file\") pod \"node-resolver-mr54p\" (UID: \"53792e6a-1284-4ac6-8370-bb836200eb67\") " pod="openshift-dns/node-resolver-mr54p"
Jul 06 18:19:29.558116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556830    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-run-netns\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.558116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556871    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-registration-dir\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.558116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.556975    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-multus-socket-dir-parent\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.558116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557000    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr7pr\" (UniqueName: \"kubernetes.io/projected/68c5ca38-8b7c-437c-a608-dc779f7b3628-kube-api-access-kr7pr\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.558116 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557018    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-sys-fs\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.558685 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557020    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-lib-modules\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.558685 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xnbw\" (UniqueName: \"kubernetes.io/projected/53792e6a-1284-4ac6-8370-bb836200eb67-kube-api-access-5xnbw\") pod \"node-resolver-mr54p\" (UID: \"53792e6a-1284-4ac6-8370-bb836200eb67\") " pod="openshift-dns/node-resolver-mr54p"
Jul 06 18:19:29.558685 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557060    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9d4a8d7b-f63e-46cc-b99b-fe9b655c3388-konnectivity-ca\") pod \"konnectivity-agent-rqp4s\" (UID: \"9d4a8d7b-f63e-46cc-b99b-fe9b655c3388\") " pod="kube-system/konnectivity-agent-rqp4s"
Jul 06 18:19:29.558685 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557069    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02f96065-7946-46c4-ad55-a5a5a0fc6520-host\") pod \"node-ca-s2ktj\" (UID: \"02f96065-7946-46c4-ad55-a5a5a0fc6520\") " pod="openshift-image-registry/node-ca-s2ktj"
Jul 06 18:19:29.558685 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557078    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-run-openvswitch\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.558685 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557062    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-var-lib-cni-bin\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.558685 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557109    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/02f96065-7946-46c4-ad55-a5a5a0fc6520-serviceca\") pod \"node-ca-s2ktj\" (UID: \"02f96065-7946-46c4-ad55-a5a5a0fc6520\") " pod="openshift-image-registry/node-ca-s2ktj"
Jul 06 18:19:29.558685 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:29.557150    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:19:29.558685 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:29.557233    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret podName:3f857b5c-ed1c-42a0-8960-524770d6f07e nodeName:}" failed. No retries permitted until 2026-07-06 18:19:30.057197007 +0000 UTC m=+4.030646499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret") pod "global-pull-secret-syncer-zdtg9" (UID: "3f857b5c-ed1c-42a0-8960-524770d6f07e") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:19:29.558685 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557239    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c840b50-b644-436f-bbb9-bcaac772988a-os-release\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.558685 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557269    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.558685 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557270    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/34b35881-3a46-4ae5-8e1b-0880314612ec-env-overrides\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.558685 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557299    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-systemd-units\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.558685 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557332    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c840b50-b644-436f-bbb9-bcaac772988a-cnibin\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557368    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c840b50-b644-436f-bbb9-bcaac772988a-cni-binary-copy\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557393    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-sysctl-conf\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557397    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/34b35881-3a46-4ae5-8e1b-0880314612ec-ovnkube-config\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557409    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-run\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557417    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-etc-kubernetes\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557426    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-var-lib-cni-multus\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557440    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-cni-bin\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557627    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d80ee81b-bb99-4520-a431-3b54ec3d9134-iptables-alerter-script\") pod \"iptables-alerter-cvntg\" (UID: \"d80ee81b-bb99-4520-a431-3b54ec3d9134\") " pod="openshift-network-operator/iptables-alerter-cvntg"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557673    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c840b50-b644-436f-bbb9-bcaac772988a-cnibin\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557757    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-run-netns\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557753    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557854    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-sys-fs\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557894    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-cni-netd\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557906    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/68c5ca38-8b7c-437c-a608-dc779f7b3628-multus-daemon-config\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557912    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/34b35881-3a46-4ae5-8e1b-0880314612ec-ovnkube-script-lib\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.559373 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:29.557930    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:19:29.559991 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557935    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0c840b50-b644-436f-bbb9-bcaac772988a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.559991 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557958    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-modprobe-d\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.559991 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557974    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-kubernetes\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.559991 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:29.557988    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs podName:a331ebbc-53c2-44a9-ab94-9c308b480dfa nodeName:}" failed. No retries permitted until 2026-07-06 18:19:30.057974244 +0000 UTC m=+4.031423742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs") pod "network-metrics-daemon-wjgsb" (UID: "a331ebbc-53c2-44a9-ab94-9c308b480dfa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:19:29.559991 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558014    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-kubernetes\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.559991 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558043    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-run-ovn-kubernetes\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.559991 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558108    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-modprobe-d\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.559991 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558194    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-etc-selinux\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.559991 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558201    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/34b35881-3a46-4ae5-8e1b-0880314612ec-ovnkube-config\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.559991 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.557674    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-multus-conf-dir\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.559991 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558255    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-os-release\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.559991 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558299    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-var-lib-cni-multus\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.559991 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-node-log\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.559991 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558379    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-log-socket\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.560536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558406    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5rbq\" (UniqueName: \"kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq\") pod \"network-check-target-rx4jm\" (UID: \"bfe5b327-4a68-43d2-90e5-ebe6e2670be0\") " pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:29.560536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558418    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-node-log\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.560536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558445    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0c840b50-b644-436f-bbb9-bcaac772988a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.560536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558450    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-log-socket\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.560536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558468    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-socket-dir\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.560536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558570    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-device-dir\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.560536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558611    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-sysctl-d\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.560536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558626    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-device-dir\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.560536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558643    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/12363263-5b1e-45b9-9ea0-c485ca8766f4-socket-dir\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.560536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558702    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-tuned\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.560536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558718    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-sysctl-d\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.560536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558779    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c840b50-b644-436f-bbb9-bcaac772988a-cni-binary-copy\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.560536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558779    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-slash\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.560536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558832    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.561198 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558842    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-slash\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.561198 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558859    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0c840b50-b644-436f-bbb9-bcaac772988a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.561198 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558868    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.561198 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558886    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3f857b5c-ed1c-42a0-8960-524770d6f07e-kubelet-config\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:29.561198 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558910    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-systemd\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.561198 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558934    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-host\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.561198 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558959    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d80ee81b-bb99-4520-a431-3b54ec3d9134-host-slash\") pod \"iptables-alerter-cvntg\" (UID: \"d80ee81b-bb99-4520-a431-3b54ec3d9134\") " pod="openshift-network-operator/iptables-alerter-cvntg"
Jul 06 18:19:29.561198 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558967    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3f857b5c-ed1c-42a0-8960-524770d6f07e-kubelet-config\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:29.561198 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.558981    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-var-lib-kubelet\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.561198 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559004    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-system-cni-dir\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.561198 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559013    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-systemd\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.561198 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559019    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-host\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.561198 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68c5ca38-8b7c-437c-a608-dc779f7b3628-cni-binary-copy\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.561198 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559062    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-var-lib-kubelet\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.561198 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559089    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-run-multus-certs\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.561728 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53792e6a-1284-4ac6-8370-bb836200eb67-tmp-dir\") pod \"node-resolver-mr54p\" (UID: \"53792e6a-1284-4ac6-8370-bb836200eb67\") " pod="openshift-dns/node-resolver-mr54p"
Jul 06 18:19:29.561728 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559131    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-system-cni-dir\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.561728 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9d4a8d7b-f63e-46cc-b99b-fe9b655c3388-agent-certs\") pod \"konnectivity-agent-rqp4s\" (UID: \"9d4a8d7b-f63e-46cc-b99b-fe9b655c3388\") " pod="kube-system/konnectivity-agent-rqp4s"
Jul 06 18:19:29.561728 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-var-lib-openvswitch\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.561728 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559195    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtsxk\" (UniqueName: \"kubernetes.io/projected/d80ee81b-bb99-4520-a431-3b54ec3d9134-kube-api-access-dtsxk\") pod \"iptables-alerter-cvntg\" (UID: \"d80ee81b-bb99-4520-a431-3b54ec3d9134\") " pod="openshift-network-operator/iptables-alerter-cvntg"
Jul 06 18:19:29.561728 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559197    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-var-lib-kubelet\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.561728 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559066    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-var-lib-kubelet\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.561728 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559220    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3f857b5c-ed1c-42a0-8960-524770d6f07e-dbus\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:29.561728 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559245    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-sysconfig\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.561728 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559249    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-var-lib-openvswitch\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.561728 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559094    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d80ee81b-bb99-4520-a431-3b54ec3d9134-host-slash\") pod \"iptables-alerter-cvntg\" (UID: \"d80ee81b-bb99-4520-a431-3b54ec3d9134\") " pod="openshift-network-operator/iptables-alerter-cvntg"
Jul 06 18:19:29.561728 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-kubelet\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.561728 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-run-ovn\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.561728 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559319    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-run-k8s-cni-cncf-io\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.561728 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559332    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0c840b50-b644-436f-bbb9-bcaac772988a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.562357 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559170    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-run-multus-certs\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.562357 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559390    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/68c5ca38-8b7c-437c-a608-dc779f7b3628-host-run-k8s-cni-cncf-io\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.562357 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559447    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-sysconfig\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.562357 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559458    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68c5ca38-8b7c-437c-a608-dc779f7b3628-cni-binary-copy\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.562357 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559490    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-host-kubelet\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.562357 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559516    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53792e6a-1284-4ac6-8370-bb836200eb67-tmp-dir\") pod \"node-resolver-mr54p\" (UID: \"53792e6a-1284-4ac6-8370-bb836200eb67\") " pod="openshift-dns/node-resolver-mr54p"
Jul 06 18:19:29.562357 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559542    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/34b35881-3a46-4ae5-8e1b-0880314612ec-run-ovn\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.562357 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.559600    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3f857b5c-ed1c-42a0-8960-524770d6f07e-dbus\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:29.562357 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.561541    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/560020e3-64da-4cfa-8cd8-178b1bd54a55-tmp\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.562357 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.561629    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/34b35881-3a46-4ae5-8e1b-0880314612ec-ovn-node-metrics-cert\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.562357 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.561814    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/560020e3-64da-4cfa-8cd8-178b1bd54a55-etc-tuned\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.562816 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.562408    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9d4a8d7b-f63e-46cc-b99b-fe9b655c3388-agent-certs\") pod \"konnectivity-agent-rqp4s\" (UID: \"9d4a8d7b-f63e-46cc-b99b-fe9b655c3388\") " pod="kube-system/konnectivity-agent-rqp4s"
Jul 06 18:19:29.568769 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:29.568745    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:19:29.568769 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:29.568771    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:19:29.568960 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:29.568785    2581 projected.go:196] Error preparing data for projected volume kube-api-access-t5rbq for pod openshift-network-diagnostics/network-check-target-rx4jm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:19:29.568960 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:29.568850    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq podName:bfe5b327-4a68-43d2-90e5-ebe6e2670be0 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:30.068833752 +0000 UTC m=+4.042283263 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-t5rbq" (UniqueName: "kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq") pod "network-check-target-rx4jm" (UID: "bfe5b327-4a68-43d2-90e5-ebe6e2670be0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:19:29.571888 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.570497    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4c4c\" (UniqueName: \"kubernetes.io/projected/a331ebbc-53c2-44a9-ab94-9c308b480dfa-kube-api-access-l4c4c\") pod \"network-metrics-daemon-wjgsb\" (UID: \"a331ebbc-53c2-44a9-ab94-9c308b480dfa\") " pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:29.573846 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.573777    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr7pr\" (UniqueName: \"kubernetes.io/projected/68c5ca38-8b7c-437c-a608-dc779f7b3628-kube-api-access-kr7pr\") pod \"multus-h4c6h\" (UID: \"68c5ca38-8b7c-437c-a608-dc779f7b3628\") " pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.574038 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.574004    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6dq4\" (UniqueName: \"kubernetes.io/projected/12363263-5b1e-45b9-9ea0-c485ca8766f4-kube-api-access-k6dq4\") pod \"aws-ebs-csi-driver-node-xxsnf\" (UID: \"12363263-5b1e-45b9-9ea0-c485ca8766f4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.574038 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.574019    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcd2c\" (UniqueName: \"kubernetes.io/projected/34b35881-3a46-4ae5-8e1b-0880314612ec-kube-api-access-jcd2c\") pod \"ovnkube-node-nv4mk\" (UID: \"34b35881-3a46-4ae5-8e1b-0880314612ec\") " pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.574486 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.574327    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xnbw\" (UniqueName: \"kubernetes.io/projected/53792e6a-1284-4ac6-8370-bb836200eb67-kube-api-access-5xnbw\") pod \"node-resolver-mr54p\" (UID: \"53792e6a-1284-4ac6-8370-bb836200eb67\") " pod="openshift-dns/node-resolver-mr54p"
Jul 06 18:19:29.574922 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.574728    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhq7x\" (UniqueName: \"kubernetes.io/projected/02f96065-7946-46c4-ad55-a5a5a0fc6520-kube-api-access-rhq7x\") pod \"node-ca-s2ktj\" (UID: \"02f96065-7946-46c4-ad55-a5a5a0fc6520\") " pod="openshift-image-registry/node-ca-s2ktj"
Jul 06 18:19:29.575019 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.574999    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtsxk\" (UniqueName: \"kubernetes.io/projected/d80ee81b-bb99-4520-a431-3b54ec3d9134-kube-api-access-dtsxk\") pod \"iptables-alerter-cvntg\" (UID: \"d80ee81b-bb99-4520-a431-3b54ec3d9134\") " pod="openshift-network-operator/iptables-alerter-cvntg"
Jul 06 18:19:29.575480 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.575461    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tww6h\" (UniqueName: \"kubernetes.io/projected/560020e3-64da-4cfa-8cd8-178b1bd54a55-kube-api-access-tww6h\") pod \"tuned-7bxlq\" (UID: \"560020e3-64da-4cfa-8cd8-178b1bd54a55\") " pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.576099 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.576078    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2744r\" (UniqueName: \"kubernetes.io/projected/0c840b50-b644-436f-bbb9-bcaac772988a-kube-api-access-2744r\") pod \"multus-additional-cni-plugins-6whc5\" (UID: \"0c840b50-b644-436f-bbb9-bcaac772988a\") " pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.612791 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.612690    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:19:29.663423 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.663387    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mr54p"
Jul 06 18:19:29.672543 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.672520    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6whc5"
Jul 06 18:19:29.675359 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:29.675275    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53792e6a_1284_4ac6_8370_bb836200eb67.slice/crio-230b5ce1c311b0915b778e557609aa0c1bcf856b78e8a7e77568ae0535a2fd39 WatchSource:0}: Error finding container 230b5ce1c311b0915b778e557609aa0c1bcf856b78e8a7e77568ae0535a2fd39: Status 404 returned error can't find the container with id 230b5ce1c311b0915b778e557609aa0c1bcf856b78e8a7e77568ae0535a2fd39
Jul 06 18:19:29.683035 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:29.683009    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c840b50_b644_436f_bbb9_bcaac772988a.slice/crio-8ddbedb7555895f0a2eec5b40a84c065f2a67bf78d417beac769a41cefc89652 WatchSource:0}: Error finding container 8ddbedb7555895f0a2eec5b40a84c065f2a67bf78d417beac769a41cefc89652: Status 404 returned error can't find the container with id 8ddbedb7555895f0a2eec5b40a84c065f2a67bf78d417beac769a41cefc89652
Jul 06 18:19:29.684225 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.684202    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cvntg"
Jul 06 18:19:29.689481 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.689415    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:29.695326 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:29.695298    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd80ee81b_bb99_4520_a431_3b54ec3d9134.slice/crio-c0d9c7b6c9e500b08881249f0e69f1925c68ccea74cb3959a1587fc5baa04611 WatchSource:0}: Error finding container c0d9c7b6c9e500b08881249f0e69f1925c68ccea74cb3959a1587fc5baa04611: Status 404 returned error can't find the container with id c0d9c7b6c9e500b08881249f0e69f1925c68ccea74cb3959a1587fc5baa04611
Jul 06 18:19:29.697512 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.697491    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rqp4s"
Jul 06 18:19:29.700149 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:29.700123    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34b35881_3a46_4ae5_8e1b_0880314612ec.slice/crio-b12195635bc9635877c567d7492e810b5ac0c94d95e9e5f9f4a0cdc015becfd2 WatchSource:0}: Error finding container b12195635bc9635877c567d7492e810b5ac0c94d95e9e5f9f4a0cdc015becfd2: Status 404 returned error can't find the container with id b12195635bc9635877c567d7492e810b5ac0c94d95e9e5f9f4a0cdc015becfd2
Jul 06 18:19:29.705130 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.705109    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7bxlq"
Jul 06 18:19:29.712891 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.712811    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h4c6h"
Jul 06 18:19:29.715374 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:29.715344    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod560020e3_64da_4cfa_8cd8_178b1bd54a55.slice/crio-2f72985da64de1fd1a8692975b0802452d971452ea717486187caa9e7e38925d WatchSource:0}: Error finding container 2f72985da64de1fd1a8692975b0802452d971452ea717486187caa9e7e38925d: Status 404 returned error can't find the container with id 2f72985da64de1fd1a8692975b0802452d971452ea717486187caa9e7e38925d
Jul 06 18:19:29.719887 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.719714    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s2ktj"
Jul 06 18:19:29.723894 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:29.723852    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68c5ca38_8b7c_437c_a608_dc779f7b3628.slice/crio-5712edc2939b9517f1cae3681c65b2eed47ee94a6df2f4bb643bb83ada6258a0 WatchSource:0}: Error finding container 5712edc2939b9517f1cae3681c65b2eed47ee94a6df2f4bb643bb83ada6258a0: Status 404 returned error can't find the container with id 5712edc2939b9517f1cae3681c65b2eed47ee94a6df2f4bb643bb83ada6258a0
Jul 06 18:19:29.726095 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:29.726073    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf"
Jul 06 18:19:29.730230 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:29.730197    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02f96065_7946_46c4_ad55_a5a5a0fc6520.slice/crio-9e75121c6e60437fcc43f68575029674881c724e9acc57c07bf006033ef173d3 WatchSource:0}: Error finding container 9e75121c6e60437fcc43f68575029674881c724e9acc57c07bf006033ef173d3: Status 404 returned error can't find the container with id 9e75121c6e60437fcc43f68575029674881c724e9acc57c07bf006033ef173d3
Jul 06 18:19:29.736427 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:19:29.736145    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12363263_5b1e_45b9_9ea0_c485ca8766f4.slice/crio-c1d232e013614ea361412a45dfd3d469034930e40693a08241d5ed92fb03c2a0 WatchSource:0}: Error finding container c1d232e013614ea361412a45dfd3d469034930e40693a08241d5ed92fb03c2a0: Status 404 returned error can't find the container with id c1d232e013614ea361412a45dfd3d469034930e40693a08241d5ed92fb03c2a0
Jul 06 18:19:30.062953 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:30.062872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:30.062953 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:30.062927    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs\") pod \"network-metrics-daemon-wjgsb\" (UID: \"a331ebbc-53c2-44a9-ab94-9c308b480dfa\") " pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:30.063171 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:30.063012    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:19:30.063171 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:30.063080    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:19:30.063171 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:30.063090    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret podName:3f857b5c-ed1c-42a0-8960-524770d6f07e nodeName:}" failed. No retries permitted until 2026-07-06 18:19:31.063070063 +0000 UTC m=+5.036519557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret") pod "global-pull-secret-syncer-zdtg9" (UID: "3f857b5c-ed1c-42a0-8960-524770d6f07e") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:19:30.063171 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:30.063130    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs podName:a331ebbc-53c2-44a9-ab94-9c308b480dfa nodeName:}" failed. No retries permitted until 2026-07-06 18:19:31.063114109 +0000 UTC m=+5.036563620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs") pod "network-metrics-daemon-wjgsb" (UID: "a331ebbc-53c2-44a9-ab94-9c308b480dfa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:19:30.163597 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:30.163560    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5rbq\" (UniqueName: \"kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq\") pod \"network-check-target-rx4jm\" (UID: \"bfe5b327-4a68-43d2-90e5-ebe6e2670be0\") " pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:30.163759 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:30.163699    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:19:30.163759 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:30.163716    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:19:30.163759 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:30.163727    2581 projected.go:196] Error preparing data for projected volume kube-api-access-t5rbq for pod openshift-network-diagnostics/network-check-target-rx4jm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:19:30.163886 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:30.163795    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq podName:bfe5b327-4a68-43d2-90e5-ebe6e2670be0 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:31.163780815 +0000 UTC m=+5.137230308 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5rbq" (UniqueName: "kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq") pod "network-check-target-rx4jm" (UID: "bfe5b327-4a68-43d2-90e5-ebe6e2670be0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:19:30.410667 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:30.410536    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 18:14:29 +0000 UTC" deadline="2028-01-08 22:07:31.108228702 +0000 UTC"
Jul 06 18:19:30.410667 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:30.410601    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13227h48m0.69763314s"
Jul 06 18:19:30.475275 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:30.475230    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s2ktj" event={"ID":"02f96065-7946-46c4-ad55-a5a5a0fc6520","Type":"ContainerStarted","Data":"9e75121c6e60437fcc43f68575029674881c724e9acc57c07bf006033ef173d3"}
Jul 06 18:19:30.476667 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:30.476636    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h4c6h" event={"ID":"68c5ca38-8b7c-437c-a608-dc779f7b3628","Type":"ContainerStarted","Data":"5712edc2939b9517f1cae3681c65b2eed47ee94a6df2f4bb643bb83ada6258a0"}
Jul 06 18:19:30.477926 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:30.477896    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cvntg" event={"ID":"d80ee81b-bb99-4520-a431-3b54ec3d9134","Type":"ContainerStarted","Data":"c0d9c7b6c9e500b08881249f0e69f1925c68ccea74cb3959a1587fc5baa04611"}
Jul 06 18:19:30.479028 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:30.479003    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mr54p" event={"ID":"53792e6a-1284-4ac6-8370-bb836200eb67","Type":"ContainerStarted","Data":"230b5ce1c311b0915b778e557609aa0c1bcf856b78e8a7e77568ae0535a2fd39"}
Jul 06 18:19:30.480344 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:30.480319    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf" event={"ID":"12363263-5b1e-45b9-9ea0-c485ca8766f4","Type":"ContainerStarted","Data":"c1d232e013614ea361412a45dfd3d469034930e40693a08241d5ed92fb03c2a0"}
Jul 06 18:19:30.481676 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:30.481630    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7bxlq" event={"ID":"560020e3-64da-4cfa-8cd8-178b1bd54a55","Type":"ContainerStarted","Data":"2f72985da64de1fd1a8692975b0802452d971452ea717486187caa9e7e38925d"}
Jul 06 18:19:30.482911 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:30.482887    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rqp4s" event={"ID":"9d4a8d7b-f63e-46cc-b99b-fe9b655c3388","Type":"ContainerStarted","Data":"d97e43f49cfc5902395633570df67af85950bc9faff422d12491ef17524ece6d"}
Jul 06 18:19:30.484276 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:30.484252    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk" event={"ID":"34b35881-3a46-4ae5-8e1b-0880314612ec","Type":"ContainerStarted","Data":"b12195635bc9635877c567d7492e810b5ac0c94d95e9e5f9f4a0cdc015becfd2"}
Jul 06 18:19:30.485364 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:30.485341    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6whc5" event={"ID":"0c840b50-b644-436f-bbb9-bcaac772988a","Type":"ContainerStarted","Data":"8ddbedb7555895f0a2eec5b40a84c065f2a67bf78d417beac769a41cefc89652"}
Jul 06 18:19:31.072077 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:31.071539    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:31.072077 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:31.071802    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:19:31.072077 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:31.071828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs\") pod \"network-metrics-daemon-wjgsb\" (UID: \"a331ebbc-53c2-44a9-ab94-9c308b480dfa\") " pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:31.072077 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:31.071881    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret podName:3f857b5c-ed1c-42a0-8960-524770d6f07e nodeName:}" failed. No retries permitted until 2026-07-06 18:19:33.071860641 +0000 UTC m=+7.045310135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret") pod "global-pull-secret-syncer-zdtg9" (UID: "3f857b5c-ed1c-42a0-8960-524770d6f07e") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:19:31.072077 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:31.071938    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:19:31.072077 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:31.071992    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs podName:a331ebbc-53c2-44a9-ab94-9c308b480dfa nodeName:}" failed. No retries permitted until 2026-07-06 18:19:33.07197483 +0000 UTC m=+7.045424327 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs") pod "network-metrics-daemon-wjgsb" (UID: "a331ebbc-53c2-44a9-ab94-9c308b480dfa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:19:31.173272 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:31.172603    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5rbq\" (UniqueName: \"kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq\") pod \"network-check-target-rx4jm\" (UID: \"bfe5b327-4a68-43d2-90e5-ebe6e2670be0\") " pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:31.173272 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:31.172813    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:19:31.173272 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:31.172834    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:19:31.173272 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:31.172846    2581 projected.go:196] Error preparing data for projected volume kube-api-access-t5rbq for pod openshift-network-diagnostics/network-check-target-rx4jm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:19:31.173272 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:31.172906    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq podName:bfe5b327-4a68-43d2-90e5-ebe6e2670be0 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:33.172887389 +0000 UTC m=+7.146336903 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5rbq" (UniqueName: "kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq") pod "network-check-target-rx4jm" (UID: "bfe5b327-4a68-43d2-90e5-ebe6e2670be0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:19:31.411269 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:31.411141    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 18:14:29 +0000 UTC" deadline="2028-03-27 10:31:50.98135756 +0000 UTC"
Jul 06 18:19:31.411269 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:31.411183    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15112h12m19.570178689s"
Jul 06 18:19:31.466366 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:31.465874    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:31.466366 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:31.465890    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:31.466366 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:31.466006    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:31.466366 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:31.466071    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:31.466366 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:31.466134    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:31.466366 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:31.466243    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:31.495785 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:31.495232    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal" event={"ID":"dd0c9e68fa1554fd01092c3bef64ba0f","Type":"ContainerStarted","Data":"7304ec2b171b86a61f3541ca4ce1ba2ec2f6bb5b0bdc6d8212f16857219c8132"}
Jul 06 18:19:33.087877 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:33.087081    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:33.087877 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:33.087133    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs\") pod \"network-metrics-daemon-wjgsb\" (UID: \"a331ebbc-53c2-44a9-ab94-9c308b480dfa\") " pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:33.087877 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:33.087273    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:19:33.087877 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:33.087335    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs podName:a331ebbc-53c2-44a9-ab94-9c308b480dfa nodeName:}" failed. No retries permitted until 2026-07-06 18:19:37.087316767 +0000 UTC m=+11.060766274 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs") pod "network-metrics-daemon-wjgsb" (UID: "a331ebbc-53c2-44a9-ab94-9c308b480dfa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:19:33.087877 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:33.087783    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:19:33.087877 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:33.087841    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret podName:3f857b5c-ed1c-42a0-8960-524770d6f07e nodeName:}" failed. No retries permitted until 2026-07-06 18:19:37.087823709 +0000 UTC m=+11.061273203 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret") pod "global-pull-secret-syncer-zdtg9" (UID: "3f857b5c-ed1c-42a0-8960-524770d6f07e") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:19:33.188973 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:33.188244    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5rbq\" (UniqueName: \"kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq\") pod \"network-check-target-rx4jm\" (UID: \"bfe5b327-4a68-43d2-90e5-ebe6e2670be0\") " pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:33.188973 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:33.188434    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:19:33.188973 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:33.188458    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:19:33.188973 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:33.188470    2581 projected.go:196] Error preparing data for projected volume kube-api-access-t5rbq for pod openshift-network-diagnostics/network-check-target-rx4jm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:19:33.188973 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:33.188527    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq podName:bfe5b327-4a68-43d2-90e5-ebe6e2670be0 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:37.188509558 +0000 UTC m=+11.161959069 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5rbq" (UniqueName: "kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq") pod "network-check-target-rx4jm" (UID: "bfe5b327-4a68-43d2-90e5-ebe6e2670be0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:19:33.468395 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:33.465841    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:33.468395 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:33.465841    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:33.468395 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:33.465853    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:33.468395 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:33.466016    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:33.468395 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:33.466139    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:33.468395 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:33.466225    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:35.466499 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:35.465646    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:35.466499 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:35.465803    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:35.466499 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:35.466207    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:35.466499 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:35.466309    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:35.466499 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:35.466372    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:35.466499 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:35.466457    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:37.124553 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:37.124510    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:37.124553 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:37.124556    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs\") pod \"network-metrics-daemon-wjgsb\" (UID: \"a331ebbc-53c2-44a9-ab94-9c308b480dfa\") " pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:37.125084 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:37.124667    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:19:37.125084 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:37.124708    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:19:37.125084 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:37.124727    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs podName:a331ebbc-53c2-44a9-ab94-9c308b480dfa nodeName:}" failed. No retries permitted until 2026-07-06 18:19:45.124705962 +0000 UTC m=+19.098155474 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs") pod "network-metrics-daemon-wjgsb" (UID: "a331ebbc-53c2-44a9-ab94-9c308b480dfa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:19:37.125084 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:37.124785    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret podName:3f857b5c-ed1c-42a0-8960-524770d6f07e nodeName:}" failed. No retries permitted until 2026-07-06 18:19:45.124767483 +0000 UTC m=+19.098216976 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret") pod "global-pull-secret-syncer-zdtg9" (UID: "3f857b5c-ed1c-42a0-8960-524770d6f07e") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:19:37.226291 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:37.225297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5rbq\" (UniqueName: \"kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq\") pod \"network-check-target-rx4jm\" (UID: \"bfe5b327-4a68-43d2-90e5-ebe6e2670be0\") " pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:37.226291 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:37.225539    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:19:37.226291 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:37.225561    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:19:37.226291 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:37.225575    2581 projected.go:196] Error preparing data for projected volume kube-api-access-t5rbq for pod openshift-network-diagnostics/network-check-target-rx4jm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:19:37.226291 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:37.225630    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq podName:bfe5b327-4a68-43d2-90e5-ebe6e2670be0 nodeName:}" failed. No retries permitted until 2026-07-06 18:19:45.225611243 +0000 UTC m=+19.199060755 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5rbq" (UniqueName: "kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq") pod "network-check-target-rx4jm" (UID: "bfe5b327-4a68-43d2-90e5-ebe6e2670be0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:19:37.467423 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:37.466797    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:37.467423 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:37.466947    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:37.467423 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:37.467072    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:37.467423 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:37.467225    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:37.467423 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:37.467273    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:37.467423 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:37.467329    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:39.465527 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:39.465498    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:39.465985 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:39.465499    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:39.465985 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:39.465631    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:39.465985 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:39.465503    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:39.465985 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:39.465777    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:39.465985 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:39.465906    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:41.465974 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:41.465940    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:41.466448 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:41.465941    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:41.466448 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:41.466067    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:41.466448 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:41.465953    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:41.466448 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:41.466150    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:41.466448 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:41.466237    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:43.466435 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:43.466394    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:43.466929 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:43.466539    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:43.466929 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:43.466616    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:43.466929 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:43.466727    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:43.466929 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:43.466805    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:43.466929 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:43.466873    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:45.188327 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:45.188291    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:45.188327 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:45.188337    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs\") pod \"network-metrics-daemon-wjgsb\" (UID: \"a331ebbc-53c2-44a9-ab94-9c308b480dfa\") " pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:45.188946 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:45.188457    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:19:45.188946 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:45.188475    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:19:45.188946 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:45.188535    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs podName:a331ebbc-53c2-44a9-ab94-9c308b480dfa nodeName:}" failed. No retries permitted until 2026-07-06 18:20:01.188513854 +0000 UTC m=+35.161963357 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs") pod "network-metrics-daemon-wjgsb" (UID: "a331ebbc-53c2-44a9-ab94-9c308b480dfa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:19:45.188946 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:45.188553    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret podName:3f857b5c-ed1c-42a0-8960-524770d6f07e nodeName:}" failed. No retries permitted until 2026-07-06 18:20:01.188545919 +0000 UTC m=+35.161995409 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret") pod "global-pull-secret-syncer-zdtg9" (UID: "3f857b5c-ed1c-42a0-8960-524770d6f07e") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:19:45.288873 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:45.288837    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5rbq\" (UniqueName: \"kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq\") pod \"network-check-target-rx4jm\" (UID: \"bfe5b327-4a68-43d2-90e5-ebe6e2670be0\") " pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:45.289066 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:45.289045    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:19:45.289125 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:45.289073    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:19:45.289125 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:45.289088    2581 projected.go:196] Error preparing data for projected volume kube-api-access-t5rbq for pod openshift-network-diagnostics/network-check-target-rx4jm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:19:45.289190 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:45.289153    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq podName:bfe5b327-4a68-43d2-90e5-ebe6e2670be0 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:01.289134569 +0000 UTC m=+35.262584082 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5rbq" (UniqueName: "kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq") pod "network-check-target-rx4jm" (UID: "bfe5b327-4a68-43d2-90e5-ebe6e2670be0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:19:45.466343 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:45.466260    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:45.466506 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:45.466260    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:45.466506 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:45.466379    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:45.466506 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:45.466480    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:45.466506 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:45.466267    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:45.466708 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:45.466591    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:47.466143 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:47.466101    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:47.466562 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:47.466202    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:47.466562 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:47.466205    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:47.466562 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:47.466319    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:47.466562 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:47.466343    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:47.466562 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:47.466493    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:49.472122 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:49.472089    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:49.472536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:49.472096    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:49.472536 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:49.472230    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:49.472536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:49.472096    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:49.472536 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:49.472286    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:49.472536 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:49.472339    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:49.542532 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:49.542501    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h4c6h" event={"ID":"68c5ca38-8b7c-437c-a608-dc779f7b3628","Type":"ContainerStarted","Data":"97460fc819700ac505ec013ed786b27e7d41f02447e80004e9b9d39e4b3a21d8"}
Jul 06 18:19:49.544039 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:49.544011    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7bxlq" event={"ID":"560020e3-64da-4cfa-8cd8-178b1bd54a55","Type":"ContainerStarted","Data":"8fe42eea7dc98a7e1cad676e9646b4b17802b7a9fbbf6a6a00f0187b65b56e38"}
Jul 06 18:19:49.545696 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:49.545676    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk" event={"ID":"34b35881-3a46-4ae5-8e1b-0880314612ec","Type":"ContainerStarted","Data":"bfdcf0344f4b59c499a64f8a1edb351515b93ad648bead1bbb165ca3cc214e9b"}
Jul 06 18:19:49.545791 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:49.545745    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk" event={"ID":"34b35881-3a46-4ae5-8e1b-0880314612ec","Type":"ContainerStarted","Data":"252df498ed04aeba7b93320fab3293cc1b8322ea5240e2f178bf9044f5f3ec34"}
Jul 06 18:19:49.574906 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:49.574866    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-229.ec2.internal" podStartSLOduration=21.574851902 podStartE2EDuration="21.574851902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:31.511686866 +0000 UTC m=+5.485136381" watchObservedRunningTime="2026-07-06 18:19:49.574851902 +0000 UTC m=+23.548301415"
Jul 06 18:19:49.575223 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:49.575203    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-h4c6h" podStartSLOduration=3.449693232 podStartE2EDuration="22.575195763s" totalImagesPullingTime="19.125502531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:49.574992376 +0000 UTC m=+23.548441875" watchObservedRunningTime="2026-07-06 18:19:49.575195763 +0000 UTC m=+23.548645275"
Jul 06 18:19:50.549549 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.549147    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mr54p" event={"ID":"53792e6a-1284-4ac6-8370-bb836200eb67","Type":"ContainerStarted","Data":"9a41fef331b407c12219003671cde0355fa76df892e031e6d910e32c44dd334e"}
Jul 06 18:19:50.550578 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.550557    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf" event={"ID":"12363263-5b1e-45b9-9ea0-c485ca8766f4","Type":"ContainerStarted","Data":"67507c23cbf91e6f1eb8369a4b1ade4da25d3a9c6050e18e901436c324e874d2"}
Jul 06 18:19:50.551990 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.551968    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rqp4s" event={"ID":"9d4a8d7b-f63e-46cc-b99b-fe9b655c3388","Type":"ContainerStarted","Data":"c1b073eb6ef9dc88e6962bac8b9b4d3fa9c64a13661a43b0f68b86ca0cefa1a9"}
Jul 06 18:19:50.554698 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.554678    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:19:50.555086 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.555065    2581 generic.go:356] "Generic (PLEG): container finished" podID="34b35881-3a46-4ae5-8e1b-0880314612ec" containerID="bfdcf0344f4b59c499a64f8a1edb351515b93ad648bead1bbb165ca3cc214e9b" exitCode=1
Jul 06 18:19:50.555170 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.555130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk" event={"ID":"34b35881-3a46-4ae5-8e1b-0880314612ec","Type":"ContainerDied","Data":"bfdcf0344f4b59c499a64f8a1edb351515b93ad648bead1bbb165ca3cc214e9b"}
Jul 06 18:19:50.555226 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.555189    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk" event={"ID":"34b35881-3a46-4ae5-8e1b-0880314612ec","Type":"ContainerStarted","Data":"66711a00411078a765ad5624ce6ff2d10e6a9e29e6611a7e3b2e3cfc9694a4b8"}
Jul 06 18:19:50.555226 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.555221    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk" event={"ID":"34b35881-3a46-4ae5-8e1b-0880314612ec","Type":"ContainerStarted","Data":"536ee09c36bc4983208ad98106756f45aee7caab0b963cb6f8f9838818fca87b"}
Jul 06 18:19:50.555302 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.555259    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk" event={"ID":"34b35881-3a46-4ae5-8e1b-0880314612ec","Type":"ContainerStarted","Data":"f1cdb15a2bba6093938bdf3734cd9e27a2159a15f5517a3cf2df779f326388d9"}
Jul 06 18:19:50.555302 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.555290    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk" event={"ID":"34b35881-3a46-4ae5-8e1b-0880314612ec","Type":"ContainerStarted","Data":"13830a1d7d8e64a2b06ecbe1fab37b20237abb7383c36aa6442b25883aec4efc"}
Jul 06 18:19:50.556486 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.556463    2581 generic.go:356] "Generic (PLEG): container finished" podID="0c840b50-b644-436f-bbb9-bcaac772988a" containerID="a197cab3ce8b2d59c04b13d7ac3ae492244cc50e244f82e4f9a1f827ca8575a1" exitCode=0
Jul 06 18:19:50.556590 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.556536    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6whc5" event={"ID":"0c840b50-b644-436f-bbb9-bcaac772988a","Type":"ContainerDied","Data":"a197cab3ce8b2d59c04b13d7ac3ae492244cc50e244f82e4f9a1f827ca8575a1"}
Jul 06 18:19:50.557939 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.557912    2581 generic.go:356] "Generic (PLEG): container finished" podID="bd320ee8d7c30c12d7b97d56aa7806c5" containerID="e1cd6cdd0a8b369e36bd8ba95a429b29509b6efee74997c82240fe339c947205" exitCode=0
Jul 06 18:19:50.557988 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.557974    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" event={"ID":"bd320ee8d7c30c12d7b97d56aa7806c5","Type":"ContainerDied","Data":"e1cd6cdd0a8b369e36bd8ba95a429b29509b6efee74997c82240fe339c947205"}
Jul 06 18:19:50.559456 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.559437    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s2ktj" event={"ID":"02f96065-7946-46c4-ad55-a5a5a0fc6520","Type":"ContainerStarted","Data":"b4f209784ba95e67153d669c11f8fc415c773e641a6e4b97e15deb1f661e3edf"}
Jul 06 18:19:50.563343 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.563282    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cvntg" event={"ID":"d80ee81b-bb99-4520-a431-3b54ec3d9134","Type":"ContainerStarted","Data":"253b608e1da621a5642187137263af56f7b461a5573169aa43bea62df0761a8b"}
Jul 06 18:19:50.567717 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.567680    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7bxlq" podStartSLOduration=4.432651504 podStartE2EDuration="23.567668722s" totalImagesPullingTime="19.135017218s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:49.604460232 +0000 UTC m=+23.577909741" watchObservedRunningTime="2026-07-06 18:19:50.567668722 +0000 UTC m=+24.541118238"
Jul 06 18:19:50.567824 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.567789    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mr54p" podStartSLOduration=4.765898854 podStartE2EDuration="23.567784175s" totalImagesPullingTime="18.801885321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:50.567292136 +0000 UTC m=+24.540741648" watchObservedRunningTime="2026-07-06 18:19:50.567784175 +0000 UTC m=+24.541233689"
Jul 06 18:19:50.627963 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.627222    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cvntg" podStartSLOduration=4.843943475 podStartE2EDuration="23.627203019s" totalImagesPullingTime="18.783259544s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:50.626962074 +0000 UTC m=+24.600411587" watchObservedRunningTime="2026-07-06 18:19:50.627203019 +0000 UTC m=+24.600652533"
Jul 06 18:19:50.641420 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.641374    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rqp4s" podStartSLOduration=4.874863852 podStartE2EDuration="23.641359674s" totalImagesPullingTime="18.766495822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:50.641153131 +0000 UTC m=+24.614602644" watchObservedRunningTime="2026-07-06 18:19:50.641359674 +0000 UTC m=+24.614809187"
Jul 06 18:19:50.655623 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:50.655580    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-s2ktj" podStartSLOduration=4.53869513 podStartE2EDuration="23.655564355s" totalImagesPullingTime="19.116869225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:50.65543878 +0000 UTC m=+24.628888293" watchObservedRunningTime="2026-07-06 18:19:50.655564355 +0000 UTC m=+24.629013867"
Jul 06 18:19:51.377713 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:51.377682    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 18:19:51.409488 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:51.409349    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T18:19:51.377709977Z","UUID":"5d57261f-2b26-400e-a6bb-c1babb7c5840","Handler":null,"Name":"","Endpoint":""}
Jul 06 18:19:51.410716 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:51.410690    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 18:19:51.410836 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:51.410747    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 18:19:51.468777 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:51.468754    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:51.468939 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:51.468754    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:51.468939 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:51.468872    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:51.468939 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:51.468754    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:51.469079 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:51.468932    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:51.469079 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:51.469006    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:51.567450 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:51.567374    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf" event={"ID":"12363263-5b1e-45b9-9ea0-c485ca8766f4","Type":"ContainerStarted","Data":"7f4ed78b3544a08ca44eff692d1feaa39f87a91df7729d21f7ca47c8ae8527eb"}
Jul 06 18:19:51.569152 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:51.569109    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" event={"ID":"bd320ee8d7c30c12d7b97d56aa7806c5","Type":"ContainerStarted","Data":"eca52e165a62e3c2bc1e7a60c1b6a060f18de1f2d939b5e7cda7f31c7e07131b"}
Jul 06 18:19:51.584009 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:51.583966    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-229.ec2.internal" podStartSLOduration=23.583952449999998 podStartE2EDuration="23.58395245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:51.583473742 +0000 UTC m=+25.556923254" watchObservedRunningTime="2026-07-06 18:19:51.58395245 +0000 UTC m=+25.557401963"
Jul 06 18:19:52.574935 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:52.574414    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf" event={"ID":"12363263-5b1e-45b9-9ea0-c485ca8766f4","Type":"ContainerStarted","Data":"2b9951187619d4400aa4f53b4d70c101272db854ec9689ec8848fd77c486992f"}
Jul 06 18:19:52.577981 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:52.577950    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:19:52.578460 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:52.578434    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk" event={"ID":"34b35881-3a46-4ae5-8e1b-0880314612ec","Type":"ContainerStarted","Data":"6f794544929a945485ed9f543fb7410548072828945fb36d436acb32261c4917"}
Jul 06 18:19:52.594143 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:52.594096    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxsnf" podStartSLOduration=4.081202452 podStartE2EDuration="25.59408206s" totalImagesPullingTime="21.512879608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:52.594014986 +0000 UTC m=+26.567464499" watchObservedRunningTime="2026-07-06 18:19:52.59408206 +0000 UTC m=+26.567531573"
Jul 06 18:19:53.011150 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:53.011116    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rqp4s"
Jul 06 18:19:53.465433 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:53.465397    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:53.465602 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:53.465401    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:53.465602 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:53.465530    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:53.465602 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:53.465578    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:53.465602 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:53.465404    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:53.465784 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:53.465659    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:54.587567 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:54.587236    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:19:54.588415 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:54.587884    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk" event={"ID":"34b35881-3a46-4ae5-8e1b-0880314612ec","Type":"ContainerStarted","Data":"672e2ebfbfae9342ff7c6534f61c001695c3d09ed6e9fba212ed3f494cb2a488"}
Jul 06 18:19:54.588415 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:54.588165    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:54.588415 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:54.588241    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:54.588415 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:54.588291    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:54.588415 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:54.588371    2581 scope.go:117] "RemoveContainer" containerID="bfdcf0344f4b59c499a64f8a1edb351515b93ad648bead1bbb165ca3cc214e9b"
Jul 06 18:19:54.604700 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:54.604572    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rqp4s"
Jul 06 18:19:54.608149 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:54.606505    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rqp4s"
Jul 06 18:19:54.611342 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:54.610972    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:54.612024 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:54.611893    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:19:55.465616 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:55.465577    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:55.465814 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:55.465590    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:55.465814 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:55.465683    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:55.465814 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:55.465697    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:55.465814 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:55.465798    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:55.466007 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:55.465889    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:55.594694 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:55.594665    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:19:55.595652 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:55.595057    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk" event={"ID":"34b35881-3a46-4ae5-8e1b-0880314612ec","Type":"ContainerStarted","Data":"1796710c8d001d421ba562cdd85e5c82a7e08b6ce917980f9be432812067014f"}
Jul 06 18:19:55.596182 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:55.596154    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rqp4s"
Jul 06 18:19:55.623685 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:55.623633    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk" podStartSLOduration=8.9870817 podStartE2EDuration="28.623606445s" totalImagesPullingTime="19.636524745s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:55.623212053 +0000 UTC m=+29.596661593" watchObservedRunningTime="2026-07-06 18:19:55.623606445 +0000 UTC m=+29.597055959"
Jul 06 18:19:56.737524 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:56.737491    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rx4jm"]
Jul 06 18:19:56.738011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:56.737603    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:56.738011 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:56.737710    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:56.740780 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:56.740629    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zdtg9"]
Jul 06 18:19:56.740780 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:56.740755    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:56.740988 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:56.740847    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:56.741494 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:56.741469    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wjgsb"]
Jul 06 18:19:56.741606 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:56.741576    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:56.741708 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:56.741688    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:58.466392 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:58.466346    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:19:58.466392 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:58.466381    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:19:58.466945 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:58.466395    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:19:58.466945 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:58.466479    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:19:58.466945 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:58.466590    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:19:58.466945 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:19:58.466664    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:19:58.603439 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:58.603407    2581 generic.go:356] "Generic (PLEG): container finished" podID="0c840b50-b644-436f-bbb9-bcaac772988a" containerID="53a2c11b5c2c81ebec72a07bc060a1bb2077d43c6253f05908803981756e7891" exitCode=0
Jul 06 18:19:58.603614 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:58.603464    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6whc5" event={"ID":"0c840b50-b644-436f-bbb9-bcaac772988a","Type":"ContainerDied","Data":"53a2c11b5c2c81ebec72a07bc060a1bb2077d43c6253f05908803981756e7891"}
Jul 06 18:19:59.614964 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:59.614924    2581 generic.go:356] "Generic (PLEG): container finished" podID="0c840b50-b644-436f-bbb9-bcaac772988a" containerID="d798c5de0c7555def604dafd533f8dfd62ad24c89cb0a557886cb0a978791e93" exitCode=0
Jul 06 18:19:59.615307 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:19:59.614977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6whc5" event={"ID":"0c840b50-b644-436f-bbb9-bcaac772988a","Type":"ContainerDied","Data":"d798c5de0c7555def604dafd533f8dfd62ad24c89cb0a557886cb0a978791e93"}
Jul 06 18:20:00.466246 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:00.466212    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:20:00.466410 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:00.466212    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:20:00.466410 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:00.466337    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:20:00.466410 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:00.466218    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:20:00.466543 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:00.466429    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:20:00.466543 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:00.466488    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:20:01.205917 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:01.205876    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:20:01.205917 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:01.205922    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs\") pod \"network-metrics-daemon-wjgsb\" (UID: \"a331ebbc-53c2-44a9-ab94-9c308b480dfa\") " pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:20:01.206386 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:01.206026    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:20:01.206386 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:01.206092    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret podName:3f857b5c-ed1c-42a0-8960-524770d6f07e nodeName:}" failed. No retries permitted until 2026-07-06 18:20:33.206076712 +0000 UTC m=+67.179526202 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret") pod "global-pull-secret-syncer-zdtg9" (UID: "3f857b5c-ed1c-42a0-8960-524770d6f07e") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:20:01.206386 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:01.206033    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:20:01.206386 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:01.206200    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs podName:a331ebbc-53c2-44a9-ab94-9c308b480dfa nodeName:}" failed. No retries permitted until 2026-07-06 18:20:33.206179147 +0000 UTC m=+67.179628656 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs") pod "network-metrics-daemon-wjgsb" (UID: "a331ebbc-53c2-44a9-ab94-9c308b480dfa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:20:01.307269 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:01.307183    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5rbq\" (UniqueName: \"kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq\") pod \"network-check-target-rx4jm\" (UID: \"bfe5b327-4a68-43d2-90e5-ebe6e2670be0\") " pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:20:01.307412 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:01.307308    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:20:01.307412 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:01.307323    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:20:01.307412 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:01.307332    2581 projected.go:196] Error preparing data for projected volume kube-api-access-t5rbq for pod openshift-network-diagnostics/network-check-target-rx4jm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:20:01.307412 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:01.307385    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq podName:bfe5b327-4a68-43d2-90e5-ebe6e2670be0 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:33.307369118 +0000 UTC m=+67.280818631 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-t5rbq" (UniqueName: "kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq") pod "network-check-target-rx4jm" (UID: "bfe5b327-4a68-43d2-90e5-ebe6e2670be0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:20:01.621978 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:01.621947    2581 generic.go:356] "Generic (PLEG): container finished" podID="0c840b50-b644-436f-bbb9-bcaac772988a" containerID="ce523277cfb9c3b5f0c1bdc59845e8600e788fb3fab99e507668cc9d53005ccc" exitCode=0
Jul 06 18:20:01.622151 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:01.622006    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6whc5" event={"ID":"0c840b50-b644-436f-bbb9-bcaac772988a","Type":"ContainerDied","Data":"ce523277cfb9c3b5f0c1bdc59845e8600e788fb3fab99e507668cc9d53005ccc"}
Jul 06 18:20:02.465827 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.465790    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:20:02.466448 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.465861    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:20:02.466448 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:02.465985    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zdtg9" podUID="3f857b5c-ed1c-42a0-8960-524770d6f07e"
Jul 06 18:20:02.466448 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.466021    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:20:02.466448 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:02.466096    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rx4jm" podUID="bfe5b327-4a68-43d2-90e5-ebe6e2670be0"
Jul 06 18:20:02.466448 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:02.466176    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wjgsb" podUID="a331ebbc-53c2-44a9-ab94-9c308b480dfa"
Jul 06 18:20:02.844251 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.844176    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-229.ec2.internal" event="NodeReady"
Jul 06 18:20:02.844386 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.844302    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 18:20:02.894194 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.894156    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tlpx5"]
Jul 06 18:20:02.907785 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.907723    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-f494fd858-w4ntf"]
Jul 06 18:20:02.907959 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.907874    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:02.912494 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.912271    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 18:20:02.912494 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.912289    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 18:20:02.912948 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.912922    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9wdv2\""
Jul 06 18:20:02.927817 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.927786    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5pcwz"]
Jul 06 18:20:02.927936 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.927876    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:02.931089 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.931065    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 18:20:02.931223 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.931127    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 18:20:02.931299 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.931243    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 18:20:02.931996 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.931925    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h42sk\""
Jul 06 18:20:02.938195 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.938175    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 18:20:02.945868 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.945848    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5pcwz"]
Jul 06 18:20:02.945976 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.945887    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tlpx5"]
Jul 06 18:20:02.945976 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.945927    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f494fd858-w4ntf"]
Jul 06 18:20:02.946063 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.945973    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5pcwz"
Jul 06 18:20:02.949523 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.949501    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 18:20:02.949632 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.949510    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 18:20:02.949854 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.949828    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 18:20:02.949944 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:02.949897    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dkzn2\""
Jul 06 18:20:03.020626 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.020589    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn9vv\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-kube-api-access-kn9vv\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.020825 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.020642    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rs29\" (UniqueName: \"kubernetes.io/projected/1c696411-a9e1-4f30-ace4-f468430c3bdf-kube-api-access-8rs29\") pod \"ingress-canary-5pcwz\" (UID: \"1c696411-a9e1-4f30-ace4-f468430c3bdf\") " pod="openshift-ingress-canary/ingress-canary-5pcwz"
Jul 06 18:20:03.020825 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.020667    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:03.020825 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.020695    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a57f230-1246-417b-b612-a3c74c8d1d57-trusted-ca\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.020825 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.020715    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-config-volume\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:03.020825 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.020783    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a57f230-1246-417b-b612-a3c74c8d1d57-installation-pull-secrets\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.020825 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.020816    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-bound-sa-token\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.021099 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.020879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vvjg\" (UniqueName: \"kubernetes.io/projected/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-kube-api-access-6vvjg\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:03.021099 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.020905    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c696411-a9e1-4f30-ace4-f468430c3bdf-cert\") pod \"ingress-canary-5pcwz\" (UID: \"1c696411-a9e1-4f30-ace4-f468430c3bdf\") " pod="openshift-ingress-canary/ingress-canary-5pcwz"
Jul 06 18:20:03.021099 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.020972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a57f230-1246-417b-b612-a3c74c8d1d57-ca-trust-extracted\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.021099 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.021013    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-certificates\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.021099 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.021046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-tmp-dir\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:03.021099 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.021069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a57f230-1246-417b-b612-a3c74c8d1d57-image-registry-private-configuration\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.021099 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.021086    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.121795 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.121765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a57f230-1246-417b-b612-a3c74c8d1d57-trusted-ca\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.121968 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.121807    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-config-volume\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:03.121968 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.121833    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a57f230-1246-417b-b612-a3c74c8d1d57-installation-pull-secrets\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.121968 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.121855    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-bound-sa-token\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.121968 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.121894    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vvjg\" (UniqueName: \"kubernetes.io/projected/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-kube-api-access-6vvjg\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:03.121968 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.121928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c696411-a9e1-4f30-ace4-f468430c3bdf-cert\") pod \"ingress-canary-5pcwz\" (UID: \"1c696411-a9e1-4f30-ace4-f468430c3bdf\") " pod="openshift-ingress-canary/ingress-canary-5pcwz"
Jul 06 18:20:03.122212 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.121985    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a57f230-1246-417b-b612-a3c74c8d1d57-ca-trust-extracted\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.122212 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.122027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-certificates\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.122212 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.122057    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-tmp-dir\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:03.122212 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.122087    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a57f230-1246-417b-b612-a3c74c8d1d57-image-registry-private-configuration\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.122212 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.122115    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.122212 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.122177    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn9vv\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-kube-api-access-kn9vv\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.122212 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.122212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rs29\" (UniqueName: \"kubernetes.io/projected/1c696411-a9e1-4f30-ace4-f468430c3bdf-kube-api-access-8rs29\") pod \"ingress-canary-5pcwz\" (UID: \"1c696411-a9e1-4f30-ace4-f468430c3bdf\") " pod="openshift-ingress-canary/ingress-canary-5pcwz"
Jul 06 18:20:03.122539 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.122243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:03.122539 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:03.122374    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:20:03.122539 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:03.122448    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls podName:9c0a0ab6-71e4-48b2-ab8c-d0540b08763d nodeName:}" failed. No retries permitted until 2026-07-06 18:20:03.622429068 +0000 UTC m=+37.595878559 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls") pod "dns-default-tlpx5" (UID: "9c0a0ab6-71e4-48b2-ab8c-d0540b08763d") : secret "dns-default-metrics-tls" not found
Jul 06 18:20:03.122539 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:03.122515    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:20:03.122539 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:03.122533    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f494fd858-w4ntf: secret "image-registry-tls" not found
Jul 06 18:20:03.122825 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:03.122593    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls podName:4a57f230-1246-417b-b612-a3c74c8d1d57 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:03.622572245 +0000 UTC m=+37.596021737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls") pod "image-registry-f494fd858-w4ntf" (UID: "4a57f230-1246-417b-b612-a3c74c8d1d57") : secret "image-registry-tls" not found
Jul 06 18:20:03.122825 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.122527    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-config-volume\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:03.122825 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.122809    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-certificates\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.122977 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.122853    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-tmp-dir\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:03.123115 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.123097    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a57f230-1246-417b-b612-a3c74c8d1d57-ca-trust-extracted\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.123496 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.123474    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a57f230-1246-417b-b612-a3c74c8d1d57-trusted-ca\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.126336 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.126313    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a57f230-1246-417b-b612-a3c74c8d1d57-installation-pull-secrets\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.126336 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.126326    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c696411-a9e1-4f30-ace4-f468430c3bdf-cert\") pod \"ingress-canary-5pcwz\" (UID: \"1c696411-a9e1-4f30-ace4-f468430c3bdf\") " pod="openshift-ingress-canary/ingress-canary-5pcwz"
Jul 06 18:20:03.126473 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.126313    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a57f230-1246-417b-b612-a3c74c8d1d57-image-registry-private-configuration\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.131156 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.131136    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vvjg\" (UniqueName: \"kubernetes.io/projected/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-kube-api-access-6vvjg\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:03.131268 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.131139    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn9vv\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-kube-api-access-kn9vv\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.131986 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.131964    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rs29\" (UniqueName: \"kubernetes.io/projected/1c696411-a9e1-4f30-ace4-f468430c3bdf-kube-api-access-8rs29\") pod \"ingress-canary-5pcwz\" (UID: \"1c696411-a9e1-4f30-ace4-f468430c3bdf\") " pod="openshift-ingress-canary/ingress-canary-5pcwz"
Jul 06 18:20:03.139398 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.139372    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-bound-sa-token\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.256226 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.256187    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5pcwz"
Jul 06 18:20:03.450441 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.450403    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5pcwz"]
Jul 06 18:20:03.455109 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:20:03.455084    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c696411_a9e1_4f30_ace4_f468430c3bdf.slice/crio-836f995b054eafe932d304e6622628878888484f9905178174d1f082e6244145 WatchSource:0}: Error finding container 836f995b054eafe932d304e6622628878888484f9905178174d1f082e6244145: Status 404 returned error can't find the container with id 836f995b054eafe932d304e6622628878888484f9905178174d1f082e6244145
Jul 06 18:20:03.625516 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.625281    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:03.625516 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.625357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:03.625516 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:03.625426    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:20:03.625516 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:03.625444    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f494fd858-w4ntf: secret "image-registry-tls" not found
Jul 06 18:20:03.625516 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:03.625465    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:20:03.625516 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:03.625511    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls podName:4a57f230-1246-417b-b612-a3c74c8d1d57 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:04.625494845 +0000 UTC m=+38.598944338 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls") pod "image-registry-f494fd858-w4ntf" (UID: "4a57f230-1246-417b-b612-a3c74c8d1d57") : secret "image-registry-tls" not found
Jul 06 18:20:03.625516 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:03.625529    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls podName:9c0a0ab6-71e4-48b2-ab8c-d0540b08763d nodeName:}" failed. No retries permitted until 2026-07-06 18:20:04.625523589 +0000 UTC m=+38.598973080 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls") pod "dns-default-tlpx5" (UID: "9c0a0ab6-71e4-48b2-ab8c-d0540b08763d") : secret "dns-default-metrics-tls" not found
Jul 06 18:20:03.628624 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:03.628595    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5pcwz" event={"ID":"1c696411-a9e1-4f30-ace4-f468430c3bdf","Type":"ContainerStarted","Data":"836f995b054eafe932d304e6622628878888484f9905178174d1f082e6244145"}
Jul 06 18:20:04.466108 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:04.466073    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:20:04.466301 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:04.466073    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:20:04.466301 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:04.466089    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:20:04.469034 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:04.469012    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 18:20:04.470440 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:04.470417    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mpmrs\""
Jul 06 18:20:04.470553 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:04.470492    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 18:20:04.470553 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:04.470544    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 18:20:04.470643 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:04.470550    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 18:20:04.470643 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:04.470621    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rgvgt\""
Jul 06 18:20:04.631835 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:04.631622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:04.632293 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:04.631788    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:20:04.632293 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:04.631901    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:04.632293 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:04.631984    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls podName:9c0a0ab6-71e4-48b2-ab8c-d0540b08763d nodeName:}" failed. No retries permitted until 2026-07-06 18:20:06.631959703 +0000 UTC m=+40.605409200 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls") pod "dns-default-tlpx5" (UID: "9c0a0ab6-71e4-48b2-ab8c-d0540b08763d") : secret "dns-default-metrics-tls" not found
Jul 06 18:20:04.632293 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:04.631998    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:20:04.632293 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:04.632008    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f494fd858-w4ntf: secret "image-registry-tls" not found
Jul 06 18:20:04.632293 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:04.632053    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls podName:4a57f230-1246-417b-b612-a3c74c8d1d57 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:06.632037187 +0000 UTC m=+40.605486695 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls") pod "image-registry-f494fd858-w4ntf" (UID: "4a57f230-1246-417b-b612-a3c74c8d1d57") : secret "image-registry-tls" not found
Jul 06 18:20:06.648550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:06.648512    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:06.648963 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:06.648565    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:06.648963 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:06.648657    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:20:06.648963 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:06.648673    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f494fd858-w4ntf: secret "image-registry-tls" not found
Jul 06 18:20:06.648963 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:06.648721    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls podName:4a57f230-1246-417b-b612-a3c74c8d1d57 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:10.64870707 +0000 UTC m=+44.622156564 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls") pod "image-registry-f494fd858-w4ntf" (UID: "4a57f230-1246-417b-b612-a3c74c8d1d57") : secret "image-registry-tls" not found
Jul 06 18:20:06.648963 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:06.648670    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:20:06.648963 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:06.648812    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls podName:9c0a0ab6-71e4-48b2-ab8c-d0540b08763d nodeName:}" failed. No retries permitted until 2026-07-06 18:20:10.648799781 +0000 UTC m=+44.622249271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls") pod "dns-default-tlpx5" (UID: "9c0a0ab6-71e4-48b2-ab8c-d0540b08763d") : secret "dns-default-metrics-tls" not found
Jul 06 18:20:07.644052 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:07.644014    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5pcwz" event={"ID":"1c696411-a9e1-4f30-ace4-f468430c3bdf","Type":"ContainerStarted","Data":"cbc0a61ed6d562f35eef73ca7ed5f00998da83f7f1c2dda05a943a69df4331b6"}
Jul 06 18:20:07.662904 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:07.662847    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5pcwz" podStartSLOduration=2.356685872 podStartE2EDuration="5.662830262s" totalImagesPullingTime="3.30614439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:07.662093668 +0000 UTC m=+41.635543181" watchObservedRunningTime="2026-07-06 18:20:07.662830262 +0000 UTC m=+41.636279773"
Jul 06 18:20:10.654013 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:10.653974    2581 generic.go:356] "Generic (PLEG): container finished" podID="0c840b50-b644-436f-bbb9-bcaac772988a" containerID="b0d7efb30521dda4d3210da2e686a85512eebaabdd4a69d918ce03952c9ad847" exitCode=0
Jul 06 18:20:10.654370 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:10.654027    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6whc5" event={"ID":"0c840b50-b644-436f-bbb9-bcaac772988a","Type":"ContainerDied","Data":"b0d7efb30521dda4d3210da2e686a85512eebaabdd4a69d918ce03952c9ad847"}
Jul 06 18:20:10.681989 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:10.681964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:10.682101 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:10.682019    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:10.682138 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:10.682110    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:20:10.682170 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:10.682114    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:20:10.682170 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:10.682163    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls podName:9c0a0ab6-71e4-48b2-ab8c-d0540b08763d nodeName:}" failed. No retries permitted until 2026-07-06 18:20:18.682145814 +0000 UTC m=+52.655595323 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls") pod "dns-default-tlpx5" (UID: "9c0a0ab6-71e4-48b2-ab8c-d0540b08763d") : secret "dns-default-metrics-tls" not found
Jul 06 18:20:10.682240 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:10.682169    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f494fd858-w4ntf: secret "image-registry-tls" not found
Jul 06 18:20:10.682240 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:10.682216    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls podName:4a57f230-1246-417b-b612-a3c74c8d1d57 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:18.682202137 +0000 UTC m=+52.655651630 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls") pod "image-registry-f494fd858-w4ntf" (UID: "4a57f230-1246-417b-b612-a3c74c8d1d57") : secret "image-registry-tls" not found
Jul 06 18:20:11.659383 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:11.659350    2581 generic.go:356] "Generic (PLEG): container finished" podID="0c840b50-b644-436f-bbb9-bcaac772988a" containerID="c5cb89feee272f03123944c89aba1d71b6b3bf3dd09bd3b23e2e06cc5713edb0" exitCode=0
Jul 06 18:20:11.659795 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:11.659402    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6whc5" event={"ID":"0c840b50-b644-436f-bbb9-bcaac772988a","Type":"ContainerDied","Data":"c5cb89feee272f03123944c89aba1d71b6b3bf3dd09bd3b23e2e06cc5713edb0"}
Jul 06 18:20:12.664977 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:12.664944    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6whc5" event={"ID":"0c840b50-b644-436f-bbb9-bcaac772988a","Type":"ContainerStarted","Data":"2bf3471706888cc7c4f64026d33587b2ff34131a56d77d05154e241e4f4069f6"}
Jul 06 18:20:12.688219 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:12.688172    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6whc5" podStartSLOduration=8.543609059 podStartE2EDuration="45.688156912s" totalImagesPullingTime="37.144547853s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:12.687509354 +0000 UTC m=+46.660958860" watchObservedRunningTime="2026-07-06 18:20:12.688156912 +0000 UTC m=+46.661606424"
Jul 06 18:20:14.424791 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.424727    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4kj2l"]
Jul 06 18:20:14.462004 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.461973    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4kj2l"]
Jul 06 18:20:14.462155 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.462089    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:14.465229 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.465205    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 18:20:14.465385 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.465250    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 18:20:14.465385 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.465287    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f7c2d\""
Jul 06 18:20:14.465385 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.465313    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 18:20:14.466485 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.466467    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 18:20:14.611536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.611505    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb45813e-f621-4f4c-b828-2b21758b7126-crio-socket\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:14.611705 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.611541    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb45813e-f621-4f4c-b828-2b21758b7126-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:14.611705 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.611583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss7wb\" (UniqueName: \"kubernetes.io/projected/eb45813e-f621-4f4c-b828-2b21758b7126-kube-api-access-ss7wb\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:14.611705 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.611610    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb45813e-f621-4f4c-b828-2b21758b7126-data-volume\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:14.611705 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.611664    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:14.712051 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.711970    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb45813e-f621-4f4c-b828-2b21758b7126-crio-socket\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:14.712051 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.712005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb45813e-f621-4f4c-b828-2b21758b7126-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:14.712051 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.712021    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss7wb\" (UniqueName: \"kubernetes.io/projected/eb45813e-f621-4f4c-b828-2b21758b7126-kube-api-access-ss7wb\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:14.712263 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:14.712102    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 18:20:14.712263 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.712125    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb45813e-f621-4f4c-b828-2b21758b7126-data-volume\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:14.712263 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:14.712168    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb45813e-f621-4f4c-b828-2b21758b7126-kube-rbac-proxy-cm podName:eb45813e-f621-4f4c-b828-2b21758b7126 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:15.212150496 +0000 UTC m=+49.185600009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb45813e-f621-4f4c-b828-2b21758b7126-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4kj2l" (UID: "eb45813e-f621-4f4c-b828-2b21758b7126") : configmap "kube-rbac-proxy" not found
Jul 06 18:20:14.712263 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.712173    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/eb45813e-f621-4f4c-b828-2b21758b7126-crio-socket\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:14.712263 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.712247    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:14.712429 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:14.712378    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:20:14.712429 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:14.712406    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls podName:eb45813e-f621-4f4c-b828-2b21758b7126 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:15.212397869 +0000 UTC m=+49.185847361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4kj2l" (UID: "eb45813e-f621-4f4c-b828-2b21758b7126") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:20:14.712494 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.712444    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/eb45813e-f621-4f4c-b828-2b21758b7126-data-volume\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:14.721932 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:14.721905    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss7wb\" (UniqueName: \"kubernetes.io/projected/eb45813e-f621-4f4c-b828-2b21758b7126-kube-api-access-ss7wb\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:15.153166 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:15.153130    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-zrd6v"]
Jul 06 18:20:15.172996 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:15.172971    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-zrd6v"]
Jul 06 18:20:15.173119 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:15.173086    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-zrd6v"
Jul 06 18:20:15.175787 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:15.175757    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 06 18:20:15.175921 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:15.175798    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 06 18:20:15.176885 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:15.176872    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-4w776\""
Jul 06 18:20:15.216766 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:15.216708    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:15.216929 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:15.216837    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb45813e-f621-4f4c-b828-2b21758b7126-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:15.216929 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:15.216849    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:20:15.216929 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:15.216905    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls podName:eb45813e-f621-4f4c-b828-2b21758b7126 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:16.216890241 +0000 UTC m=+50.190339731 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4kj2l" (UID: "eb45813e-f621-4f4c-b828-2b21758b7126") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:20:15.216929 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:15.216920    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 18:20:15.217095 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:15.216968    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb45813e-f621-4f4c-b828-2b21758b7126-kube-rbac-proxy-cm podName:eb45813e-f621-4f4c-b828-2b21758b7126 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:16.216953082 +0000 UTC m=+50.190402577 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb45813e-f621-4f4c-b828-2b21758b7126-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4kj2l" (UID: "eb45813e-f621-4f4c-b828-2b21758b7126") : configmap "kube-rbac-proxy" not found
Jul 06 18:20:15.317436 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:15.317401    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4z7p\" (UniqueName: \"kubernetes.io/projected/8cf57350-15ce-4895-9903-7bec2d51230d-kube-api-access-m4z7p\") pod \"migrator-d57d58cc9-zrd6v\" (UID: \"8cf57350-15ce-4895-9903-7bec2d51230d\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-zrd6v"
Jul 06 18:20:15.417948 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:15.417880    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4z7p\" (UniqueName: \"kubernetes.io/projected/8cf57350-15ce-4895-9903-7bec2d51230d-kube-api-access-m4z7p\") pod \"migrator-d57d58cc9-zrd6v\" (UID: \"8cf57350-15ce-4895-9903-7bec2d51230d\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-zrd6v"
Jul 06 18:20:15.426080 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:15.426050    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4z7p\" (UniqueName: \"kubernetes.io/projected/8cf57350-15ce-4895-9903-7bec2d51230d-kube-api-access-m4z7p\") pod \"migrator-d57d58cc9-zrd6v\" (UID: \"8cf57350-15ce-4895-9903-7bec2d51230d\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-zrd6v"
Jul 06 18:20:15.481420 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:15.481388    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-zrd6v"
Jul 06 18:20:15.613853 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:15.612318    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-zrd6v"]
Jul 06 18:20:15.617438 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:20:15.617407    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cf57350_15ce_4895_9903_7bec2d51230d.slice/crio-a7a6c4afa04b90fffac62b63e3bc033881afeab20d72641dbf3d6b4902c72b6f WatchSource:0}: Error finding container a7a6c4afa04b90fffac62b63e3bc033881afeab20d72641dbf3d6b4902c72b6f: Status 404 returned error can't find the container with id a7a6c4afa04b90fffac62b63e3bc033881afeab20d72641dbf3d6b4902c72b6f
Jul 06 18:20:15.673353 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:15.673290    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-zrd6v" event={"ID":"8cf57350-15ce-4895-9903-7bec2d51230d","Type":"ContainerStarted","Data":"a7a6c4afa04b90fffac62b63e3bc033881afeab20d72641dbf3d6b4902c72b6f"}
Jul 06 18:20:16.027833 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:16.027764    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mr54p_53792e6a-1284-4ac6-8370-bb836200eb67/dns-node-resolver/0.log"
Jul 06 18:20:16.222599 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:16.222541    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb45813e-f621-4f4c-b828-2b21758b7126-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:16.222729 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:16.222666    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 18:20:16.222729 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:16.222715    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:16.222828 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:16.222746    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/eb45813e-f621-4f4c-b828-2b21758b7126-kube-rbac-proxy-cm podName:eb45813e-f621-4f4c-b828-2b21758b7126 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:18.222714843 +0000 UTC m=+52.196164334 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/eb45813e-f621-4f4c-b828-2b21758b7126-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4kj2l" (UID: "eb45813e-f621-4f4c-b828-2b21758b7126") : configmap "kube-rbac-proxy" not found
Jul 06 18:20:16.222828 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:16.222815    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:20:16.222894 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:16.222868    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls podName:eb45813e-f621-4f4c-b828-2b21758b7126 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:18.222857137 +0000 UTC m=+52.196306632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4kj2l" (UID: "eb45813e-f621-4f4c-b828-2b21758b7126") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:20:17.028401 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.028373    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s2ktj_02f96065-7946-46c4-ad55-a5a5a0fc6520/node-ca/0.log"
Jul 06 18:20:17.342768 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.342675    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xsqtt"]
Jul 06 18:20:17.353786 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.353756    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xsqtt"]
Jul 06 18:20:17.353954 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.353888    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xsqtt"
Jul 06 18:20:17.356649 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.356621    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-pxp44\""
Jul 06 18:20:17.356649 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.356649    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 06 18:20:17.358449 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.357967    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 06 18:20:17.358449 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.358205    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 06 18:20:17.358835 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.358654    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 06 18:20:17.532840 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.532807    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8fe3469b-6c74-4ca6-a0b2-630512efda04-signing-cabundle\") pod \"service-ca-758f599c5-xsqtt\" (UID: \"8fe3469b-6c74-4ca6-a0b2-630512efda04\") " pod="openshift-service-ca/service-ca-758f599c5-xsqtt"
Jul 06 18:20:17.532840 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.532842    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w247\" (UniqueName: \"kubernetes.io/projected/8fe3469b-6c74-4ca6-a0b2-630512efda04-kube-api-access-9w247\") pod \"service-ca-758f599c5-xsqtt\" (UID: \"8fe3469b-6c74-4ca6-a0b2-630512efda04\") " pod="openshift-service-ca/service-ca-758f599c5-xsqtt"
Jul 06 18:20:17.533022 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.532908    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8fe3469b-6c74-4ca6-a0b2-630512efda04-signing-key\") pod \"service-ca-758f599c5-xsqtt\" (UID: \"8fe3469b-6c74-4ca6-a0b2-630512efda04\") " pod="openshift-service-ca/service-ca-758f599c5-xsqtt"
Jul 06 18:20:17.628698 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.628666    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5pcwz_1c696411-a9e1-4f30-ace4-f468430c3bdf/serve-healthcheck-canary/0.log"
Jul 06 18:20:17.633851 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.633832    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8fe3469b-6c74-4ca6-a0b2-630512efda04-signing-key\") pod \"service-ca-758f599c5-xsqtt\" (UID: \"8fe3469b-6c74-4ca6-a0b2-630512efda04\") " pod="openshift-service-ca/service-ca-758f599c5-xsqtt"
Jul 06 18:20:17.634000 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.633987    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8fe3469b-6c74-4ca6-a0b2-630512efda04-signing-cabundle\") pod \"service-ca-758f599c5-xsqtt\" (UID: \"8fe3469b-6c74-4ca6-a0b2-630512efda04\") " pod="openshift-service-ca/service-ca-758f599c5-xsqtt"
Jul 06 18:20:17.634035 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.634010    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w247\" (UniqueName: \"kubernetes.io/projected/8fe3469b-6c74-4ca6-a0b2-630512efda04-kube-api-access-9w247\") pod \"service-ca-758f599c5-xsqtt\" (UID: \"8fe3469b-6c74-4ca6-a0b2-630512efda04\") " pod="openshift-service-ca/service-ca-758f599c5-xsqtt"
Jul 06 18:20:17.634568 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.634544    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8fe3469b-6c74-4ca6-a0b2-630512efda04-signing-cabundle\") pod \"service-ca-758f599c5-xsqtt\" (UID: \"8fe3469b-6c74-4ca6-a0b2-630512efda04\") " pod="openshift-service-ca/service-ca-758f599c5-xsqtt"
Jul 06 18:20:17.636209 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.636183    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8fe3469b-6c74-4ca6-a0b2-630512efda04-signing-key\") pod \"service-ca-758f599c5-xsqtt\" (UID: \"8fe3469b-6c74-4ca6-a0b2-630512efda04\") " pod="openshift-service-ca/service-ca-758f599c5-xsqtt"
Jul 06 18:20:17.642626 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.642599    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w247\" (UniqueName: \"kubernetes.io/projected/8fe3469b-6c74-4ca6-a0b2-630512efda04-kube-api-access-9w247\") pod \"service-ca-758f599c5-xsqtt\" (UID: \"8fe3469b-6c74-4ca6-a0b2-630512efda04\") " pod="openshift-service-ca/service-ca-758f599c5-xsqtt"
Jul 06 18:20:17.666446 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.666424    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-xsqtt"
Jul 06 18:20:17.860556 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:17.860449    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-xsqtt"]
Jul 06 18:20:17.862959 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:20:17.862932    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fe3469b_6c74_4ca6_a0b2_630512efda04.slice/crio-ff14379c393db5acc6dc55957926082cfb415da2df04a94b0527817a73ffe613 WatchSource:0}: Error finding container ff14379c393db5acc6dc55957926082cfb415da2df04a94b0527817a73ffe613: Status 404 returned error can't find the container with id ff14379c393db5acc6dc55957926082cfb415da2df04a94b0527817a73ffe613
Jul 06 18:20:18.239203 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:18.239115    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb45813e-f621-4f4c-b828-2b21758b7126-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:18.239203 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:18.239183    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:18.239652 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:18.239270    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:20:18.239652 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:18.239319    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls podName:eb45813e-f621-4f4c-b828-2b21758b7126 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:22.239305803 +0000 UTC m=+56.212755294 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4kj2l" (UID: "eb45813e-f621-4f4c-b828-2b21758b7126") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:20:18.239753 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:18.239665    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/eb45813e-f621-4f4c-b828-2b21758b7126-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:18.681489 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:18.681453    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-zrd6v" event={"ID":"8cf57350-15ce-4895-9903-7bec2d51230d","Type":"ContainerStarted","Data":"529cbd8bdc6a152bea971786a452520b7e0278f8683c07dc098486d021fee206"}
Jul 06 18:20:18.681756 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:18.681519    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-zrd6v" event={"ID":"8cf57350-15ce-4895-9903-7bec2d51230d","Type":"ContainerStarted","Data":"7ad837c26894e2155c3c9269d151da3fe24b8af0f93840a379b041947c3c3c6a"}
Jul 06 18:20:18.682538 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:18.682510    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xsqtt" event={"ID":"8fe3469b-6c74-4ca6-a0b2-630512efda04","Type":"ContainerStarted","Data":"ff14379c393db5acc6dc55957926082cfb415da2df04a94b0527817a73ffe613"}
Jul 06 18:20:18.700914 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:18.700853    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-zrd6v" podStartSLOduration=1.579413197 podStartE2EDuration="3.700836261s" totalImagesPullingTime="2.121423064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:18.699503001 +0000 UTC m=+52.672952520" watchObservedRunningTime="2026-07-06 18:20:18.700836261 +0000 UTC m=+52.674285773"
Jul 06 18:20:18.742020 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:18.741991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:18.742161 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:18.742063    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:18.742203 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:18.742164    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:20:18.742203 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:18.742178    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:20:18.742203 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:18.742180    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f494fd858-w4ntf: secret "image-registry-tls" not found
Jul 06 18:20:18.742302 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:18.742234    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls podName:4a57f230-1246-417b-b612-a3c74c8d1d57 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:34.742219591 +0000 UTC m=+68.715669095 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls") pod "image-registry-f494fd858-w4ntf" (UID: "4a57f230-1246-417b-b612-a3c74c8d1d57") : secret "image-registry-tls" not found
Jul 06 18:20:18.742302 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:18.742273    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls podName:9c0a0ab6-71e4-48b2-ab8c-d0540b08763d nodeName:}" failed. No retries permitted until 2026-07-06 18:20:34.742242642 +0000 UTC m=+68.715692132 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls") pod "dns-default-tlpx5" (UID: "9c0a0ab6-71e4-48b2-ab8c-d0540b08763d") : secret "dns-default-metrics-tls" not found
Jul 06 18:20:21.694361 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:21.694318    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-xsqtt" event={"ID":"8fe3469b-6c74-4ca6-a0b2-630512efda04","Type":"ContainerStarted","Data":"49a7ffc2af6440528789dcb3f07f57cb9333ab22b98094156b7986b831a0ea7d"}
Jul 06 18:20:21.710953 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:21.710899    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-xsqtt" podStartSLOduration=1.926017096 podStartE2EDuration="4.710878377s" totalImagesPullingTime="2.784861281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:21.710818795 +0000 UTC m=+55.684268308" watchObservedRunningTime="2026-07-06 18:20:21.710878377 +0000 UTC m=+55.684327889"
Jul 06 18:20:22.270154 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:22.270110    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:22.270347 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:22.270268    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:20:22.270347 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:22.270333    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls podName:eb45813e-f621-4f4c-b828-2b21758b7126 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:30.270318789 +0000 UTC m=+64.243768280 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4kj2l" (UID: "eb45813e-f621-4f4c-b828-2b21758b7126") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:20:26.608699 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:26.608666    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nv4mk"
Jul 06 18:20:30.332015 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:30.331972    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:30.334418 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:30.334379    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/eb45813e-f621-4f4c-b828-2b21758b7126-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4kj2l\" (UID: \"eb45813e-f621-4f4c-b828-2b21758b7126\") " pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:30.373672 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:30.373645    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f7c2d\""
Jul 06 18:20:30.381647 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:30.381625    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4kj2l"
Jul 06 18:20:30.534240 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:30.534207    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4kj2l"]
Jul 06 18:20:30.537231 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:20:30.537195    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb45813e_f621_4f4c_b828_2b21758b7126.slice/crio-67dc564797804cf319e56e5f23398b46825476c059591555fba87bf3b5208f47 WatchSource:0}: Error finding container 67dc564797804cf319e56e5f23398b46825476c059591555fba87bf3b5208f47: Status 404 returned error can't find the container with id 67dc564797804cf319e56e5f23398b46825476c059591555fba87bf3b5208f47
Jul 06 18:20:30.719479 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:30.719441    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4kj2l" event={"ID":"eb45813e-f621-4f4c-b828-2b21758b7126","Type":"ContainerStarted","Data":"67dc564797804cf319e56e5f23398b46825476c059591555fba87bf3b5208f47"}
Jul 06 18:20:33.261310 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:33.261267    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:20:33.261784 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:33.261321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs\") pod \"network-metrics-daemon-wjgsb\" (UID: \"a331ebbc-53c2-44a9-ab94-9c308b480dfa\") " pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:20:33.263958 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:33.263929    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 18:20:33.265040 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:33.265010    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 18:20:33.274514 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:33.274493    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3f857b5c-ed1c-42a0-8960-524770d6f07e-original-pull-secret\") pod \"global-pull-secret-syncer-zdtg9\" (UID: \"3f857b5c-ed1c-42a0-8960-524770d6f07e\") " pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:20:33.274618 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:33.274537    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a331ebbc-53c2-44a9-ab94-9c308b480dfa-metrics-certs\") pod \"network-metrics-daemon-wjgsb\" (UID: \"a331ebbc-53c2-44a9-ab94-9c308b480dfa\") " pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:20:33.284709 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:33.284689    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zdtg9"
Jul 06 18:20:33.292416 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:33.292398    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rgvgt\""
Jul 06 18:20:33.300414 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:33.300400    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wjgsb"
Jul 06 18:20:33.362803 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:33.362770    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5rbq\" (UniqueName: \"kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq\") pod \"network-check-target-rx4jm\" (UID: \"bfe5b327-4a68-43d2-90e5-ebe6e2670be0\") " pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:20:33.365544 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:33.365527    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 18:20:33.376235 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:33.376212    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 18:20:33.386706 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:33.386677    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5rbq\" (UniqueName: \"kubernetes.io/projected/bfe5b327-4a68-43d2-90e5-ebe6e2670be0-kube-api-access-t5rbq\") pod \"network-check-target-rx4jm\" (UID: \"bfe5b327-4a68-43d2-90e5-ebe6e2670be0\") " pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:20:33.580702 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:33.580627    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mpmrs\""
Jul 06 18:20:33.588140 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:33.588114    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:20:34.401438 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:34.401406    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zdtg9"]
Jul 06 18:20:34.404694 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:20:34.404671    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f857b5c_ed1c_42a0_8960_524770d6f07e.slice/crio-369d64748df0bec1578ed77d003b34fed05e0d38154197dc8fb2e8667c00784d WatchSource:0}: Error finding container 369d64748df0bec1578ed77d003b34fed05e0d38154197dc8fb2e8667c00784d: Status 404 returned error can't find the container with id 369d64748df0bec1578ed77d003b34fed05e0d38154197dc8fb2e8667c00784d
Jul 06 18:20:34.621161 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:34.621129    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rx4jm"]
Jul 06 18:20:34.624090 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:34.624067    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wjgsb"]
Jul 06 18:20:34.624234 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:20:34.624211    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe5b327_4a68_43d2_90e5_ebe6e2670be0.slice/crio-ddfaf7be481b73704115ef0ebdbe6bbcd4c04da67468b37fd70f6492e4ea4afd WatchSource:0}: Error finding container ddfaf7be481b73704115ef0ebdbe6bbcd4c04da67468b37fd70f6492e4ea4afd: Status 404 returned error can't find the container with id ddfaf7be481b73704115ef0ebdbe6bbcd4c04da67468b37fd70f6492e4ea4afd
Jul 06 18:20:34.627623 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:20:34.627574    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda331ebbc_53c2_44a9_ab94_9c308b480dfa.slice/crio-34c10e3b5beb48cf55c42b29074c4d1bf10fb85b26161758fa59b8cfd950d4f4 WatchSource:0}: Error finding container 34c10e3b5beb48cf55c42b29074c4d1bf10fb85b26161758fa59b8cfd950d4f4: Status 404 returned error can't find the container with id 34c10e3b5beb48cf55c42b29074c4d1bf10fb85b26161758fa59b8cfd950d4f4
Jul 06 18:20:34.734330 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:34.734287    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4kj2l" event={"ID":"eb45813e-f621-4f4c-b828-2b21758b7126","Type":"ContainerStarted","Data":"403c76bbc9415ad75ce4825be7ae30e3d5b65e635419f0aae2a8eee4c7917c9b"}
Jul 06 18:20:34.735341 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:34.735315    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rx4jm" event={"ID":"bfe5b327-4a68-43d2-90e5-ebe6e2670be0","Type":"ContainerStarted","Data":"ddfaf7be481b73704115ef0ebdbe6bbcd4c04da67468b37fd70f6492e4ea4afd"}
Jul 06 18:20:34.736441 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:34.736418    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zdtg9" event={"ID":"3f857b5c-ed1c-42a0-8960-524770d6f07e","Type":"ContainerStarted","Data":"369d64748df0bec1578ed77d003b34fed05e0d38154197dc8fb2e8667c00784d"}
Jul 06 18:20:34.737382 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:34.737365    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wjgsb" event={"ID":"a331ebbc-53c2-44a9-ab94-9c308b480dfa","Type":"ContainerStarted","Data":"34c10e3b5beb48cf55c42b29074c4d1bf10fb85b26161758fa59b8cfd950d4f4"}
Jul 06 18:20:34.777148 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:34.777109    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:34.777308 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:34.777211    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:34.779692 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:34.779663    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c0a0ab6-71e4-48b2-ab8c-d0540b08763d-metrics-tls\") pod \"dns-default-tlpx5\" (UID: \"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d\") " pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:34.779853 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:34.779837    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls\") pod \"image-registry-f494fd858-w4ntf\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") " pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:35.022983 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.022944    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9wdv2\""
Jul 06 18:20:35.031323 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.031295    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:35.040618 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.040587    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h42sk\""
Jul 06 18:20:35.049190 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.049157    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:35.217929 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.216593    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tlpx5"]
Jul 06 18:20:35.226577 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:20:35.226547    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c0a0ab6_71e4_48b2_ab8c_d0540b08763d.slice/crio-42bc3b81fd67a6791720a9606742624cdfd078ee38a798d9a9cd33add2f1b3c1 WatchSource:0}: Error finding container 42bc3b81fd67a6791720a9606742624cdfd078ee38a798d9a9cd33add2f1b3c1: Status 404 returned error can't find the container with id 42bc3b81fd67a6791720a9606742624cdfd078ee38a798d9a9cd33add2f1b3c1
Jul 06 18:20:35.249320 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.249291    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f494fd858-w4ntf"]
Jul 06 18:20:35.253444 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:20:35.253414    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a57f230_1246_417b_b612_a3c74c8d1d57.slice/crio-acd1c7043739e5ea702faad98f6e284bbb54b348da7427549a5c8888cab28380 WatchSource:0}: Error finding container acd1c7043739e5ea702faad98f6e284bbb54b348da7427549a5c8888cab28380: Status 404 returned error can't find the container with id acd1c7043739e5ea702faad98f6e284bbb54b348da7427549a5c8888cab28380
Jul 06 18:20:35.746802 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.745910    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4kj2l" event={"ID":"eb45813e-f621-4f4c-b828-2b21758b7126","Type":"ContainerStarted","Data":"1fd72d304d1c1f3a2ad215a76c824e343aa80958ab9db884986acb158ec256cd"}
Jul 06 18:20:35.748239 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.748206    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f494fd858-w4ntf" event={"ID":"4a57f230-1246-417b-b612-a3c74c8d1d57","Type":"ContainerStarted","Data":"2d626b400625676c63d38e004b532c815e0f53ff97ab35efc4d062c9529dab44"}
Jul 06 18:20:35.748367 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.748253    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f494fd858-w4ntf" event={"ID":"4a57f230-1246-417b-b612-a3c74c8d1d57","Type":"ContainerStarted","Data":"acd1c7043739e5ea702faad98f6e284bbb54b348da7427549a5c8888cab28380"}
Jul 06 18:20:35.748367 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.748311    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:20:35.750414 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.750374    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tlpx5" event={"ID":"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d","Type":"ContainerStarted","Data":"42bc3b81fd67a6791720a9606742624cdfd078ee38a798d9a9cd33add2f1b3c1"}
Jul 06 18:20:35.979263 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.979202    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-f494fd858-w4ntf" podStartSLOduration=64.979178946 podStartE2EDuration="1m4.979178946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:19:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:35.770765109 +0000 UTC m=+69.744214622" watchObservedRunningTime="2026-07-06 18:20:35.979178946 +0000 UTC m=+69.952628457"
Jul 06 18:20:35.981565 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.981528    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5688bbb7c4-57gbg"]
Jul 06 18:20:35.986118 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.986079    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"]
Jul 06 18:20:35.986247 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.986187    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5688bbb7c4-57gbg"
Jul 06 18:20:35.989605 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.989567    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 06 18:20:35.989971 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.989937    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"]
Jul 06 18:20:35.990704 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.990682    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"
Jul 06 18:20:35.998129 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.997647    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 06 18:20:35.998129 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.998047    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 06 18:20:35.998358 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.998338    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 06 18:20:35.998658 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:35.998638    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-v8gs8\""
Jul 06 18:20:36.003242 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.001066    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.005349 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.004826    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5688bbb7c4-57gbg"]
Jul 06 18:20:36.007062 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.006900    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 06 18:20:36.007863 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.007297    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 06 18:20:36.007863 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.007511    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"]
Jul 06 18:20:36.007863 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.007536    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 06 18:20:36.014968 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.013210    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 06 18:20:36.014968 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.013552    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 06 18:20:36.014968 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.014889    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"]
Jul 06 18:20:36.047256 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.047182    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f494fd858-w4ntf"]
Jul 06 18:20:36.087090 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.087030    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-ca\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.087090 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.087077    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.087323 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.087103    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.087323 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.087133    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/749d10fd-2619-48df-9228-5318913cb32d-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f45cc4976-b5wkc\" (UID: \"749d10fd-2619-48df-9228-5318913cb32d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"
Jul 06 18:20:36.087323 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.087157    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.087323 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.087179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/abe74d31-791d-4bf1-8c60-1ae6787bf4d3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5688bbb7c4-57gbg\" (UID: \"abe74d31-791d-4bf1-8c60-1ae6787bf4d3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5688bbb7c4-57gbg"
Jul 06 18:20:36.087323 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.087208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/749d10fd-2619-48df-9228-5318913cb32d-tmp\") pod \"klusterlet-addon-workmgr-7f45cc4976-b5wkc\" (UID: \"749d10fd-2619-48df-9228-5318913cb32d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"
Jul 06 18:20:36.087323 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.087247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8dn6\" (UniqueName: \"kubernetes.io/projected/749d10fd-2619-48df-9228-5318913cb32d-kube-api-access-w8dn6\") pod \"klusterlet-addon-workmgr-7f45cc4976-b5wkc\" (UID: \"749d10fd-2619-48df-9228-5318913cb32d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"
Jul 06 18:20:36.087323 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.087288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-hub\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.087323 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.087311    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v44bg\" (UniqueName: \"kubernetes.io/projected/abe74d31-791d-4bf1-8c60-1ae6787bf4d3-kube-api-access-v44bg\") pod \"managed-serviceaccount-addon-agent-5688bbb7c4-57gbg\" (UID: \"abe74d31-791d-4bf1-8c60-1ae6787bf4d3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5688bbb7c4-57gbg"
Jul 06 18:20:36.087724 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.087352    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvhzk\" (UniqueName: \"kubernetes.io/projected/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-kube-api-access-wvhzk\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.188895 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.188655    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.188895 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.188702    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/749d10fd-2619-48df-9228-5318913cb32d-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f45cc4976-b5wkc\" (UID: \"749d10fd-2619-48df-9228-5318913cb32d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"
Jul 06 18:20:36.188895 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.188750    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.188895 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.188774    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/abe74d31-791d-4bf1-8c60-1ae6787bf4d3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5688bbb7c4-57gbg\" (UID: \"abe74d31-791d-4bf1-8c60-1ae6787bf4d3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5688bbb7c4-57gbg"
Jul 06 18:20:36.191828 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.190374    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/749d10fd-2619-48df-9228-5318913cb32d-tmp\") pod \"klusterlet-addon-workmgr-7f45cc4976-b5wkc\" (UID: \"749d10fd-2619-48df-9228-5318913cb32d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"
Jul 06 18:20:36.191828 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.190440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8dn6\" (UniqueName: \"kubernetes.io/projected/749d10fd-2619-48df-9228-5318913cb32d-kube-api-access-w8dn6\") pod \"klusterlet-addon-workmgr-7f45cc4976-b5wkc\" (UID: \"749d10fd-2619-48df-9228-5318913cb32d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"
Jul 06 18:20:36.191828 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.190487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-hub\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.191828 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.190512    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v44bg\" (UniqueName: \"kubernetes.io/projected/abe74d31-791d-4bf1-8c60-1ae6787bf4d3-kube-api-access-v44bg\") pod \"managed-serviceaccount-addon-agent-5688bbb7c4-57gbg\" (UID: \"abe74d31-791d-4bf1-8c60-1ae6787bf4d3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5688bbb7c4-57gbg"
Jul 06 18:20:36.191828 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.190540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvhzk\" (UniqueName: \"kubernetes.io/projected/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-kube-api-access-wvhzk\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.191828 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.190574    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-ca\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.191828 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.190599    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.191828 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.190770    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/749d10fd-2619-48df-9228-5318913cb32d-tmp\") pod \"klusterlet-addon-workmgr-7f45cc4976-b5wkc\" (UID: \"749d10fd-2619-48df-9228-5318913cb32d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"
Jul 06 18:20:36.191828 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.191263    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-additional-service-ca\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.195934 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.195460    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-hub\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.197853 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.197809    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-ca\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.198626 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.198583    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.199360 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.199293    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/749d10fd-2619-48df-9228-5318913cb32d-klusterlet-config\") pod \"klusterlet-addon-workmgr-7f45cc4976-b5wkc\" (UID: \"749d10fd-2619-48df-9228-5318913cb32d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"
Jul 06 18:20:36.199457 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.199420    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/abe74d31-791d-4bf1-8c60-1ae6787bf4d3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5688bbb7c4-57gbg\" (UID: \"abe74d31-791d-4bf1-8c60-1ae6787bf4d3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5688bbb7c4-57gbg"
Jul 06 18:20:36.201097 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.201058    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.207640 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.207579    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8dn6\" (UniqueName: \"kubernetes.io/projected/749d10fd-2619-48df-9228-5318913cb32d-kube-api-access-w8dn6\") pod \"klusterlet-addon-workmgr-7f45cc4976-b5wkc\" (UID: \"749d10fd-2619-48df-9228-5318913cb32d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"
Jul 06 18:20:36.209832 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.209783    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvhzk\" (UniqueName: \"kubernetes.io/projected/fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9-kube-api-access-wvhzk\") pod \"cluster-proxy-proxy-agent-8776df756-q4z4x\" (UID: \"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.211244 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.211202    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v44bg\" (UniqueName: \"kubernetes.io/projected/abe74d31-791d-4bf1-8c60-1ae6787bf4d3-kube-api-access-v44bg\") pod \"managed-serviceaccount-addon-agent-5688bbb7c4-57gbg\" (UID: \"abe74d31-791d-4bf1-8c60-1ae6787bf4d3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5688bbb7c4-57gbg"
Jul 06 18:20:36.352927 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.352833    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5688bbb7c4-57gbg"
Jul 06 18:20:36.365661 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.365392    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"
Jul 06 18:20:36.375422 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.375192    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"
Jul 06 18:20:36.555653 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.555594    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5688bbb7c4-57gbg"]
Jul 06 18:20:36.572916 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:20:36.572859    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabe74d31_791d_4bf1_8c60_1ae6787bf4d3.slice/crio-1b18e9e483516b844ecbbef68601589b32fd09e18e61e049f5d960054f984dd1 WatchSource:0}: Error finding container 1b18e9e483516b844ecbbef68601589b32fd09e18e61e049f5d960054f984dd1: Status 404 returned error can't find the container with id 1b18e9e483516b844ecbbef68601589b32fd09e18e61e049f5d960054f984dd1
Jul 06 18:20:36.582305 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.582282    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"]
Jul 06 18:20:36.585708 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:20:36.585680    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749d10fd_2619_48df_9228_5318913cb32d.slice/crio-610f552f00d7ea6a231176deecce4f8cf9d28c023a2b83aeb858dfd8d1e48566 WatchSource:0}: Error finding container 610f552f00d7ea6a231176deecce4f8cf9d28c023a2b83aeb858dfd8d1e48566: Status 404 returned error can't find the container with id 610f552f00d7ea6a231176deecce4f8cf9d28c023a2b83aeb858dfd8d1e48566
Jul 06 18:20:36.603197 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.603095    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x"]
Jul 06 18:20:36.615705 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:20:36.615645    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa149a35_8bd3_4d7a_8f7a_bb48315cc8f9.slice/crio-5db89ea7867d87da19696df7b0e86c1688506045645fedd703e23e5495581bdf WatchSource:0}: Error finding container 5db89ea7867d87da19696df7b0e86c1688506045645fedd703e23e5495581bdf: Status 404 returned error can't find the container with id 5db89ea7867d87da19696df7b0e86c1688506045645fedd703e23e5495581bdf
Jul 06 18:20:36.782914 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.782874    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wjgsb" event={"ID":"a331ebbc-53c2-44a9-ab94-9c308b480dfa","Type":"ContainerStarted","Data":"6fa3b30871b2d327798672e08b9ae908756dae0e6b4bf1c65c241fa0c3248231"}
Jul 06 18:20:36.783376 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.782948    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wjgsb" event={"ID":"a331ebbc-53c2-44a9-ab94-9c308b480dfa","Type":"ContainerStarted","Data":"7b159933e466a634e693c8ee908ebaec56423475d2b85846dbccae2cee4a4cf6"}
Jul 06 18:20:36.785560 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.785532    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x" event={"ID":"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9","Type":"ContainerStarted","Data":"5db89ea7867d87da19696df7b0e86c1688506045645fedd703e23e5495581bdf"}
Jul 06 18:20:36.787773 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.787699    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5688bbb7c4-57gbg" event={"ID":"abe74d31-791d-4bf1-8c60-1ae6787bf4d3","Type":"ContainerStarted","Data":"1b18e9e483516b844ecbbef68601589b32fd09e18e61e049f5d960054f984dd1"}
Jul 06 18:20:36.789704 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.789679    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc" event={"ID":"749d10fd-2619-48df-9228-5318913cb32d","Type":"ContainerStarted","Data":"610f552f00d7ea6a231176deecce4f8cf9d28c023a2b83aeb858dfd8d1e48566"}
Jul 06 18:20:36.801123 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:36.800592    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wjgsb" podStartSLOduration=68.678168772 podStartE2EDuration="1m9.800573422s" totalImagesPullingTime="1.12240465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:36.80013973 +0000 UTC m=+70.773589243" watchObservedRunningTime="2026-07-06 18:20:36.800573422 +0000 UTC m=+70.774022937"
Jul 06 18:20:41.815761 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:41.815344    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4kj2l" event={"ID":"eb45813e-f621-4f4c-b828-2b21758b7126","Type":"ContainerStarted","Data":"0d1b813620c02fedc704acb344b0f0b1540c6388bea27e66ba61de5ed0776592"}
Jul 06 18:20:41.818989 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:41.818956    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rx4jm" event={"ID":"bfe5b327-4a68-43d2-90e5-ebe6e2670be0","Type":"ContainerStarted","Data":"61a83d0bdaa521375bc4a28638867cf3a2028d30daba48f6b5be4b80a8e3c27a"}
Jul 06 18:20:41.819292 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:41.819239    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:20:41.821006 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:41.820979    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zdtg9" event={"ID":"3f857b5c-ed1c-42a0-8960-524770d6f07e","Type":"ContainerStarted","Data":"72f5e20b36e5d7979c22e74caef362caeec4d3953fafed63bc956f60c062fce1"}
Jul 06 18:20:41.836079 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:41.832811    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tlpx5" event={"ID":"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d","Type":"ContainerStarted","Data":"dc1836e04415c79c5b5a8037f5ccfa1826dab212a15bc12583ad2e92713de488"}
Jul 06 18:20:41.836079 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:41.832915    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tlpx5" event={"ID":"9c0a0ab6-71e4-48b2-ab8c-d0540b08763d","Type":"ContainerStarted","Data":"c642f682c1c2fca6818952ec9e9c187f261dd16e4a1bd3aca62db614f09ca39f"}
Jul 06 18:20:41.836079 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:41.833723    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:41.837528 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:41.837469    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4kj2l" podStartSLOduration=17.076644976 podStartE2EDuration="27.837450183s" totalImagesPullingTime="10.760805207s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:20:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:41.834285228 +0000 UTC m=+75.807734756" watchObservedRunningTime="2026-07-06 18:20:41.837450183 +0000 UTC m=+75.810899697"
Jul 06 18:20:41.872579 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:41.872522    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rx4jm" podStartSLOduration=68.061808463 podStartE2EDuration="1m14.872502775s" totalImagesPullingTime="6.810694312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:41.852911344 +0000 UTC m=+75.826360857" watchObservedRunningTime="2026-07-06 18:20:41.872502775 +0000 UTC m=+75.845952283"
Jul 06 18:20:41.872846 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:41.872799    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tlpx5" podStartSLOduration=33.676088235 podStartE2EDuration="39.872789932s" totalImagesPullingTime="6.196701697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:41.87148732 +0000 UTC m=+75.844936835" watchObservedRunningTime="2026-07-06 18:20:41.872789932 +0000 UTC m=+75.846239468"
Jul 06 18:20:41.889439 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:41.889374    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zdtg9" podStartSLOduration=67.857199394 podStartE2EDuration="1m14.889354053s" totalImagesPullingTime="7.032154659s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:19:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:41.889068647 +0000 UTC m=+75.862518161" watchObservedRunningTime="2026-07-06 18:20:41.889354053 +0000 UTC m=+75.862803566"
Jul 06 18:20:45.849349 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:45.849320    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x" event={"ID":"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9","Type":"ContainerStarted","Data":"a7a09992e5d0b0ac6c36ef0dcf3a8ee971c1d0c2a2b4ba73b80394a169e4b760"}
Jul 06 18:20:45.849722 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:45.849363    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x" event={"ID":"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9","Type":"ContainerStarted","Data":"bea7d35f0faffaec5be644ae785105a429bbab34e43a9b8e6fc70513c88864ec"}
Jul 06 18:20:45.850925 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:45.850896    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc" event={"ID":"749d10fd-2619-48df-9228-5318913cb32d","Type":"ContainerStarted","Data":"00da78ce54409f2a34e9760317bfe56449febaacddb3f5e6e8e7f58c34fa77e8"}
Jul 06 18:20:45.851122 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:45.851092    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"
Jul 06 18:20:45.852923 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:45.852903    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc"
Jul 06 18:20:45.866990 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:45.866946    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7f45cc4976-b5wkc" podStartSLOduration=1.783800915 podStartE2EDuration="10.866932116s" totalImagesPullingTime="9.083131201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:45.866132666 +0000 UTC m=+79.839582195" watchObservedRunningTime="2026-07-06 18:20:45.866932116 +0000 UTC m=+79.840381633"
Jul 06 18:20:46.857934 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:46.857896    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x" event={"ID":"fa149a35-8bd3-4d7a-8f7a-bb48315cc8f9","Type":"ContainerStarted","Data":"b5ed004989e978304202c2ab1526d0c8a9b9ff2b97cdf9251be1309c422b00ad"}
Jul 06 18:20:46.879144 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:46.879089    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8776df756-q4z4x" podStartSLOduration=2.8243188999999997 podStartE2EDuration="11.879073726s" totalImagesPullingTime="9.054754826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:46.877195713 +0000 UTC m=+80.850645226" watchObservedRunningTime="2026-07-06 18:20:46.879073726 +0000 UTC m=+80.852523287"
Jul 06 18:20:51.042694 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.042584    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h5f5c"]
Jul 06 18:20:51.048851 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.048830    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.052246 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.052222    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 18:20:51.052377 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.052326    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 18:20:51.053495 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.053477    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 18:20:51.053640 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.053550    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 18:20:51.053760 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.053563    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 18:20:51.053809 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.053597    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5f786\""
Jul 06 18:20:51.053922 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.053905    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 18:20:51.119998 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.119976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-tls\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.120128 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.120007    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-textfile\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.120128 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.120027    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.120128 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.120054    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-accelerators-collector-config\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.120286 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.120161    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-sys\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.120286 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.120199    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-root\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.120286 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.120244    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-metrics-client-ca\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.120286 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.120272    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-wtmp\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.120471 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.120306    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpjfd\" (UniqueName: \"kubernetes.io/projected/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-kube-api-access-vpjfd\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.221431 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.221386    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-textfile\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.221621 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.221432    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.221621 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.221497    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-accelerators-collector-config\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.221621 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.221543    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-sys\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.221621 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.221572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-root\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.221857 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.221653    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-root\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.221857 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.221690    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-metrics-client-ca\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.221857 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.221717    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-wtmp\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.221857 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.221767    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpjfd\" (UniqueName: \"kubernetes.io/projected/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-kube-api-access-vpjfd\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.221857 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.221803    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-tls\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.222088 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:51.221915    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 06 18:20:51.222088 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.221923    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-wtmp\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.222088 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:20:51.221981    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-tls podName:1ca034ed-9fa2-4244-921f-29e1aa2b7a53 nodeName:}" failed. No retries permitted until 2026-07-06 18:20:51.721956123 +0000 UTC m=+85.695405619 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-tls") pod "node-exporter-h5f5c" (UID: "1ca034ed-9fa2-4244-921f-29e1aa2b7a53") : secret "node-exporter-tls" not found
Jul 06 18:20:51.222260 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.221714    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-sys\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.222260 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.222186    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-accelerators-collector-config\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.222375 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.222265    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-metrics-client-ca\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.222428 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.222415    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-textfile\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.224496 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.224469    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.244297 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.244265    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpjfd\" (UniqueName: \"kubernetes.io/projected/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-kube-api-access-vpjfd\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.727339 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.727300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-tls\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.730058 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.730034    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1ca034ed-9fa2-4244-921f-29e1aa2b7a53-node-exporter-tls\") pod \"node-exporter-h5f5c\" (UID: \"1ca034ed-9fa2-4244-921f-29e1aa2b7a53\") " pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.960358 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:51.960330    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h5f5c"
Jul 06 18:20:51.972571 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:20:51.972541    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ca034ed_9fa2_4244_921f_29e1aa2b7a53.slice/crio-9c41bda5bea6482304cd95cb648e60b19f00f4503c9e6b5fc24fc134c87114dd WatchSource:0}: Error finding container 9c41bda5bea6482304cd95cb648e60b19f00f4503c9e6b5fc24fc134c87114dd: Status 404 returned error can't find the container with id 9c41bda5bea6482304cd95cb648e60b19f00f4503c9e6b5fc24fc134c87114dd
Jul 06 18:20:52.843548 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:52.843459    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tlpx5"
Jul 06 18:20:52.879665 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:52.879618    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5688bbb7c4-57gbg" event={"ID":"abe74d31-791d-4bf1-8c60-1ae6787bf4d3","Type":"ContainerStarted","Data":"d1bf52d315bd0d0ff905f6241a6711dc68a922387ab2cf0b1313675ef5decc4a"}
Jul 06 18:20:52.881070 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:52.881043    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h5f5c" event={"ID":"1ca034ed-9fa2-4244-921f-29e1aa2b7a53","Type":"ContainerStarted","Data":"9c41bda5bea6482304cd95cb648e60b19f00f4503c9e6b5fc24fc134c87114dd"}
Jul 06 18:20:52.899242 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:52.899196    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5688bbb7c4-57gbg" podStartSLOduration=2.016328761 podStartE2EDuration="17.89917336s" totalImagesPullingTime="15.882844599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:52.898438532 +0000 UTC m=+86.871888044" watchObservedRunningTime="2026-07-06 18:20:52.89917336 +0000 UTC m=+86.872622876"
Jul 06 18:20:53.886124 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:53.886086    2581 generic.go:356] "Generic (PLEG): container finished" podID="1ca034ed-9fa2-4244-921f-29e1aa2b7a53" containerID="9817db7f70f3da3316ab506c5645318727fd348b7b6fdc1700ec55699d2ee1d4" exitCode=0
Jul 06 18:20:53.886535 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:53.886177    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h5f5c" event={"ID":"1ca034ed-9fa2-4244-921f-29e1aa2b7a53","Type":"ContainerDied","Data":"9817db7f70f3da3316ab506c5645318727fd348b7b6fdc1700ec55699d2ee1d4"}
Jul 06 18:20:54.891750 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:54.891710    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h5f5c" event={"ID":"1ca034ed-9fa2-4244-921f-29e1aa2b7a53","Type":"ContainerStarted","Data":"6899dc0e8e618765b4ed1385c3bd227121d0d9709d54d769c9570240a469e946"}
Jul 06 18:20:54.892128 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:54.891821    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h5f5c" event={"ID":"1ca034ed-9fa2-4244-921f-29e1aa2b7a53","Type":"ContainerStarted","Data":"76ed78ac93eda4fd9eda04179c8ac0d038278fc9e480e79103a7a5cecda9774a"}
Jul 06 18:20:54.911438 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:54.911391    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h5f5c" podStartSLOduration=2.64944928 podStartE2EDuration="3.911377671s" totalImagesPullingTime="1.198815193s" totalInitContainerRuntime="63.113198ms" isStatefulPod=true podCreationTimestamp="2026-07-06 18:20:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:20:54.909498422 +0000 UTC m=+88.882947949" watchObservedRunningTime="2026-07-06 18:20:54.911377671 +0000 UTC m=+88.884827184"
Jul 06 18:20:57.802904 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:20:57.802877    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:21:02.825847 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:02.825801    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-f494fd858-w4ntf" podUID="4a57f230-1246-417b-b612-a3c74c8d1d57" containerName="registry" containerID="cri-o://2d626b400625676c63d38e004b532c815e0f53ff97ab35efc4d062c9529dab44" gracePeriod=30
Jul 06 18:21:03.071372 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.071347    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:21:03.217214 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.217183    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-bound-sa-token\") pod \"4a57f230-1246-417b-b612-a3c74c8d1d57\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") "
Jul 06 18:21:03.217407 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.217248    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a57f230-1246-417b-b612-a3c74c8d1d57-ca-trust-extracted\") pod \"4a57f230-1246-417b-b612-a3c74c8d1d57\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") "
Jul 06 18:21:03.217407 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.217287    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-certificates\") pod \"4a57f230-1246-417b-b612-a3c74c8d1d57\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") "
Jul 06 18:21:03.217407 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.217315    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a57f230-1246-417b-b612-a3c74c8d1d57-installation-pull-secrets\") pod \"4a57f230-1246-417b-b612-a3c74c8d1d57\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") "
Jul 06 18:21:03.217407 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.217340    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a57f230-1246-417b-b612-a3c74c8d1d57-trusted-ca\") pod \"4a57f230-1246-417b-b612-a3c74c8d1d57\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") "
Jul 06 18:21:03.217407 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.217356    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a57f230-1246-417b-b612-a3c74c8d1d57-image-registry-private-configuration\") pod \"4a57f230-1246-417b-b612-a3c74c8d1d57\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") "
Jul 06 18:21:03.217407 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.217378    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kn9vv\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-kube-api-access-kn9vv\") pod \"4a57f230-1246-417b-b612-a3c74c8d1d57\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") "
Jul 06 18:21:03.217407 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.217405    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls\") pod \"4a57f230-1246-417b-b612-a3c74c8d1d57\" (UID: \"4a57f230-1246-417b-b612-a3c74c8d1d57\") "
Jul 06 18:21:03.217870 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.217803    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4a57f230-1246-417b-b612-a3c74c8d1d57" (UID: "4a57f230-1246-417b-b612-a3c74c8d1d57"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:21:03.218129 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.218076    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a57f230-1246-417b-b612-a3c74c8d1d57-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4a57f230-1246-417b-b612-a3c74c8d1d57" (UID: "4a57f230-1246-417b-b612-a3c74c8d1d57"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:21:03.220250 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.220193    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a57f230-1246-417b-b612-a3c74c8d1d57-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4a57f230-1246-417b-b612-a3c74c8d1d57" (UID: "4a57f230-1246-417b-b612-a3c74c8d1d57"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:21:03.220347 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.220288    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-kube-api-access-kn9vv" (OuterVolumeSpecName: "kube-api-access-kn9vv") pod "4a57f230-1246-417b-b612-a3c74c8d1d57" (UID: "4a57f230-1246-417b-b612-a3c74c8d1d57"). InnerVolumeSpecName "kube-api-access-kn9vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:21:03.220409 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.220346    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4a57f230-1246-417b-b612-a3c74c8d1d57" (UID: "4a57f230-1246-417b-b612-a3c74c8d1d57"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:21:03.220464 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.220413    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4a57f230-1246-417b-b612-a3c74c8d1d57" (UID: "4a57f230-1246-417b-b612-a3c74c8d1d57"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:21:03.220464 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.220422    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a57f230-1246-417b-b612-a3c74c8d1d57-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4a57f230-1246-417b-b612-a3c74c8d1d57" (UID: "4a57f230-1246-417b-b612-a3c74c8d1d57"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:21:03.226223 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.226201    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a57f230-1246-417b-b612-a3c74c8d1d57-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4a57f230-1246-417b-b612-a3c74c8d1d57" (UID: "4a57f230-1246-417b-b612-a3c74c8d1d57"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:21:03.318689 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.318656    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a57f230-1246-417b-b612-a3c74c8d1d57-trusted-ca\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:21:03.318689 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.318686    2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a57f230-1246-417b-b612-a3c74c8d1d57-image-registry-private-configuration\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:21:03.318899 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.318698    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn9vv\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-kube-api-access-kn9vv\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:21:03.318899 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.318708    2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-tls\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:21:03.318899 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.318716    2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a57f230-1246-417b-b612-a3c74c8d1d57-bound-sa-token\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:21:03.318899 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.318724    2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a57f230-1246-417b-b612-a3c74c8d1d57-ca-trust-extracted\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:21:03.318899 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.318750    2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a57f230-1246-417b-b612-a3c74c8d1d57-registry-certificates\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:21:03.318899 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.318758    2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a57f230-1246-417b-b612-a3c74c8d1d57-installation-pull-secrets\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:21:03.922120 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.922085    2581 generic.go:356] "Generic (PLEG): container finished" podID="4a57f230-1246-417b-b612-a3c74c8d1d57" containerID="2d626b400625676c63d38e004b532c815e0f53ff97ab35efc4d062c9529dab44" exitCode=0
Jul 06 18:21:03.922549 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.922138    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f494fd858-w4ntf" event={"ID":"4a57f230-1246-417b-b612-a3c74c8d1d57","Type":"ContainerDied","Data":"2d626b400625676c63d38e004b532c815e0f53ff97ab35efc4d062c9529dab44"}
Jul 06 18:21:03.922549 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.922145    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f494fd858-w4ntf"
Jul 06 18:21:03.922549 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.922187    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f494fd858-w4ntf" event={"ID":"4a57f230-1246-417b-b612-a3c74c8d1d57","Type":"ContainerDied","Data":"acd1c7043739e5ea702faad98f6e284bbb54b348da7427549a5c8888cab28380"}
Jul 06 18:21:03.922794 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.922779    2581 scope.go:117] "RemoveContainer" containerID="2d626b400625676c63d38e004b532c815e0f53ff97ab35efc4d062c9529dab44"
Jul 06 18:21:03.931357 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.931337    2581 scope.go:117] "RemoveContainer" containerID="2d626b400625676c63d38e004b532c815e0f53ff97ab35efc4d062c9529dab44"
Jul 06 18:21:03.931656 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:21:03.931636    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d626b400625676c63d38e004b532c815e0f53ff97ab35efc4d062c9529dab44\": container with ID starting with 2d626b400625676c63d38e004b532c815e0f53ff97ab35efc4d062c9529dab44 not found: ID does not exist" containerID="2d626b400625676c63d38e004b532c815e0f53ff97ab35efc4d062c9529dab44"
Jul 06 18:21:03.931716 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.931663    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d626b400625676c63d38e004b532c815e0f53ff97ab35efc4d062c9529dab44"} err="failed to get container status \"2d626b400625676c63d38e004b532c815e0f53ff97ab35efc4d062c9529dab44\": rpc error: code = NotFound desc = could not find container \"2d626b400625676c63d38e004b532c815e0f53ff97ab35efc4d062c9529dab44\": container with ID starting with 2d626b400625676c63d38e004b532c815e0f53ff97ab35efc4d062c9529dab44 not found: ID does not exist"
Jul 06 18:21:03.938442 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.938423    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f494fd858-w4ntf"]
Jul 06 18:21:03.943510 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:03.943490    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-f494fd858-w4ntf"]
Jul 06 18:21:05.472928 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:05.472897    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a57f230-1246-417b-b612-a3c74c8d1d57" path="/var/lib/kubelet/pods/4a57f230-1246-417b-b612-a3c74c8d1d57/volumes"
Jul 06 18:21:12.839084 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:21:12.839054    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rx4jm"
Jul 06 18:24:01.847404 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:01.847319    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftdgm"]
Jul 06 18:24:01.847935 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:01.847593    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a57f230-1246-417b-b612-a3c74c8d1d57" containerName="registry"
Jul 06 18:24:01.847935 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:01.847606    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a57f230-1246-417b-b612-a3c74c8d1d57" containerName="registry"
Jul 06 18:24:01.847935 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:01.847646    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a57f230-1246-417b-b612-a3c74c8d1d57" containerName="registry"
Jul 06 18:24:01.850339 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:01.850323    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftdgm"
Jul 06 18:24:01.856817 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:01.853040    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 06 18:24:01.856817 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:01.854136    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 06 18:24:01.856817 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:01.854417    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-g8fm6\""
Jul 06 18:24:01.859834 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:01.859811    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftdgm"]
Jul 06 18:24:01.981691 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:01.981654    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc9g6\" (UniqueName: \"kubernetes.io/projected/99b0df35-7d84-45be-9bf2-c7bcc6ffc225-kube-api-access-cc9g6\") pod \"openshift-lws-operator-fd8ccff4c-ftdgm\" (UID: \"99b0df35-7d84-45be-9bf2-c7bcc6ffc225\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftdgm"
Jul 06 18:24:01.981691 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:01.981693    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99b0df35-7d84-45be-9bf2-c7bcc6ffc225-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ftdgm\" (UID: \"99b0df35-7d84-45be-9bf2-c7bcc6ffc225\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftdgm"
Jul 06 18:24:02.082990 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:02.082956    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc9g6\" (UniqueName: \"kubernetes.io/projected/99b0df35-7d84-45be-9bf2-c7bcc6ffc225-kube-api-access-cc9g6\") pod \"openshift-lws-operator-fd8ccff4c-ftdgm\" (UID: \"99b0df35-7d84-45be-9bf2-c7bcc6ffc225\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftdgm"
Jul 06 18:24:02.082990 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:02.082996    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99b0df35-7d84-45be-9bf2-c7bcc6ffc225-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ftdgm\" (UID: \"99b0df35-7d84-45be-9bf2-c7bcc6ffc225\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftdgm"
Jul 06 18:24:02.083332 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:02.083317    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99b0df35-7d84-45be-9bf2-c7bcc6ffc225-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ftdgm\" (UID: \"99b0df35-7d84-45be-9bf2-c7bcc6ffc225\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftdgm"
Jul 06 18:24:02.091101 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:02.091069    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc9g6\" (UniqueName: \"kubernetes.io/projected/99b0df35-7d84-45be-9bf2-c7bcc6ffc225-kube-api-access-cc9g6\") pod \"openshift-lws-operator-fd8ccff4c-ftdgm\" (UID: \"99b0df35-7d84-45be-9bf2-c7bcc6ffc225\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftdgm"
Jul 06 18:24:02.165014 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:02.164981    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftdgm"
Jul 06 18:24:02.296574 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:02.296532    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftdgm"]
Jul 06 18:24:02.299392 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:24:02.299357    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99b0df35_7d84_45be_9bf2_c7bcc6ffc225.slice/crio-7e19646142f8f1711ca886cd307a91d6e91d301b68496adb7221cc7309c19db0 WatchSource:0}: Error finding container 7e19646142f8f1711ca886cd307a91d6e91d301b68496adb7221cc7309c19db0: Status 404 returned error can't find the container with id 7e19646142f8f1711ca886cd307a91d6e91d301b68496adb7221cc7309c19db0
Jul 06 18:24:02.522505 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:02.522421    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftdgm" event={"ID":"99b0df35-7d84-45be-9bf2-c7bcc6ffc225","Type":"ContainerStarted","Data":"7e19646142f8f1711ca886cd307a91d6e91d301b68496adb7221cc7309c19db0"}
Jul 06 18:24:04.530842 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:04.530805    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftdgm" event={"ID":"99b0df35-7d84-45be-9bf2-c7bcc6ffc225","Type":"ContainerStarted","Data":"b4be3bf938c0c4d56b1c5a221deb36846a444bf5a5e14f01178b59876abcf31d"}
Jul 06 18:24:04.546887 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:04.546821    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ftdgm" podStartSLOduration=1.389082802 podStartE2EDuration="3.546799785s" totalImagesPullingTime="2.157716983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:04.546441171 +0000 UTC m=+278.519890686" watchObservedRunningTime="2026-07-06 18:24:04.546799785 +0000 UTC m=+278.520249297"
Jul 06 18:24:21.242581 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.242536    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"]
Jul 06 18:24:21.245553 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.245534    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.249142 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.249116    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 06 18:24:21.249279 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.249202    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 06 18:24:21.249381 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.249353    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 06 18:24:21.250051 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.250034    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 06 18:24:21.250289 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.250275    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mm5zn\""
Jul 06 18:24:21.261131 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.261106    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"]
Jul 06 18:24:21.329609 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.329573    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/306ee2d9-ff73-4216-9768-d718aee079a2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bf95b557-574t4\" (UID: \"306ee2d9-ff73-4216-9768-d718aee079a2\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.329609 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.329607    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/306ee2d9-ff73-4216-9768-d718aee079a2-manifests\") pod \"opendatahub-operator-controller-manager-bf95b557-574t4\" (UID: \"306ee2d9-ff73-4216-9768-d718aee079a2\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.329867 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.329635    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/306ee2d9-ff73-4216-9768-d718aee079a2-webhook-cert\") pod \"opendatahub-operator-controller-manager-bf95b557-574t4\" (UID: \"306ee2d9-ff73-4216-9768-d718aee079a2\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.329867 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.329779    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/306ee2d9-ff73-4216-9768-d718aee079a2-tmp\") pod \"opendatahub-operator-controller-manager-bf95b557-574t4\" (UID: \"306ee2d9-ff73-4216-9768-d718aee079a2\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.329867 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.329815    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgkbs\" (UniqueName: \"kubernetes.io/projected/306ee2d9-ff73-4216-9768-d718aee079a2-kube-api-access-jgkbs\") pod \"opendatahub-operator-controller-manager-bf95b557-574t4\" (UID: \"306ee2d9-ff73-4216-9768-d718aee079a2\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.430288 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.430250    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/306ee2d9-ff73-4216-9768-d718aee079a2-webhook-cert\") pod \"opendatahub-operator-controller-manager-bf95b557-574t4\" (UID: \"306ee2d9-ff73-4216-9768-d718aee079a2\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.430288 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.430293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/306ee2d9-ff73-4216-9768-d718aee079a2-tmp\") pod \"opendatahub-operator-controller-manager-bf95b557-574t4\" (UID: \"306ee2d9-ff73-4216-9768-d718aee079a2\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.430516 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.430312    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgkbs\" (UniqueName: \"kubernetes.io/projected/306ee2d9-ff73-4216-9768-d718aee079a2-kube-api-access-jgkbs\") pod \"opendatahub-operator-controller-manager-bf95b557-574t4\" (UID: \"306ee2d9-ff73-4216-9768-d718aee079a2\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.430516 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.430367    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/306ee2d9-ff73-4216-9768-d718aee079a2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bf95b557-574t4\" (UID: \"306ee2d9-ff73-4216-9768-d718aee079a2\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.430516 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.430390    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/306ee2d9-ff73-4216-9768-d718aee079a2-manifests\") pod \"opendatahub-operator-controller-manager-bf95b557-574t4\" (UID: \"306ee2d9-ff73-4216-9768-d718aee079a2\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.430831 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.430808    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/306ee2d9-ff73-4216-9768-d718aee079a2-manifests\") pod \"opendatahub-operator-controller-manager-bf95b557-574t4\" (UID: \"306ee2d9-ff73-4216-9768-d718aee079a2\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.432837 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.432811    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/306ee2d9-ff73-4216-9768-d718aee079a2-tmp\") pod \"opendatahub-operator-controller-manager-bf95b557-574t4\" (UID: \"306ee2d9-ff73-4216-9768-d718aee079a2\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.432997 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.432979    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/306ee2d9-ff73-4216-9768-d718aee079a2-webhook-cert\") pod \"opendatahub-operator-controller-manager-bf95b557-574t4\" (UID: \"306ee2d9-ff73-4216-9768-d718aee079a2\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.433292 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.433274    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/306ee2d9-ff73-4216-9768-d718aee079a2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bf95b557-574t4\" (UID: \"306ee2d9-ff73-4216-9768-d718aee079a2\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.443024 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.442995    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgkbs\" (UniqueName: \"kubernetes.io/projected/306ee2d9-ff73-4216-9768-d718aee079a2-kube-api-access-jgkbs\") pod \"opendatahub-operator-controller-manager-bf95b557-574t4\" (UID: \"306ee2d9-ff73-4216-9768-d718aee079a2\") " pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.555256 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.555174    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:21.692272 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:21.692244    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"]
Jul 06 18:24:21.695029 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:24:21.695001    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306ee2d9_ff73_4216_9768_d718aee079a2.slice/crio-1e24cd3d126ccc1025184fc6f8c87e792a7d8481b2741c9834e21ad0484ce1e9 WatchSource:0}: Error finding container 1e24cd3d126ccc1025184fc6f8c87e792a7d8481b2741c9834e21ad0484ce1e9: Status 404 returned error can't find the container with id 1e24cd3d126ccc1025184fc6f8c87e792a7d8481b2741c9834e21ad0484ce1e9
Jul 06 18:24:22.591683 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:22.591610    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4" event={"ID":"306ee2d9-ff73-4216-9768-d718aee079a2","Type":"ContainerStarted","Data":"1e24cd3d126ccc1025184fc6f8c87e792a7d8481b2741c9834e21ad0484ce1e9"}
Jul 06 18:24:24.602384 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:24.602349    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4" event={"ID":"306ee2d9-ff73-4216-9768-d718aee079a2","Type":"ContainerStarted","Data":"64c2ed1b51bf18c9a192f262d63be3c8974164045a3162ea5e19846084a59432"}
Jul 06 18:24:25.607393 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:25.607363    2581 generic.go:356] "Generic (PLEG): container finished" podID="306ee2d9-ff73-4216-9768-d718aee079a2" containerID="64c2ed1b51bf18c9a192f262d63be3c8974164045a3162ea5e19846084a59432" exitCode=0
Jul 06 18:24:25.607783 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:25.607432    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4" event={"ID":"306ee2d9-ff73-4216-9768-d718aee079a2","Type":"ContainerDied","Data":"64c2ed1b51bf18c9a192f262d63be3c8974164045a3162ea5e19846084a59432"}
Jul 06 18:24:26.613536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:26.613498    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4" event={"ID":"306ee2d9-ff73-4216-9768-d718aee079a2","Type":"ContainerStarted","Data":"e13cb289f03c6d6805fb2babfe4fadc2f58eb803a92cde2b9a9b9b5f82ade81d"}
Jul 06 18:24:26.614030 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:26.613655    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:26.635239 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:26.635181    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4" podStartSLOduration=1.426884977 podStartE2EDuration="5.635162981s" totalImagesPullingTime="3.197103398s" totalInitContainerRuntime="1.011174606s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:26.634830127 +0000 UTC m=+300.608279631" watchObservedRunningTime="2026-07-06 18:24:26.635162981 +0000 UTC m=+300.608612493"
Jul 06 18:24:27.364293 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:27.364264    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:24:27.365719 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:27.365690    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:24:27.370228 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:27.370207    2581 kubelet.go:1648] "Image garbage collection succeeded"
Jul 06 18:24:37.619901 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:37.619870    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-bf95b557-574t4"
Jul 06 18:24:41.718653 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:41.718604    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ghrjn"]
Jul 06 18:24:41.727144 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:41.727117    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ghrjn"
Jul 06 18:24:41.730095 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:41.730069    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 06 18:24:41.730425 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:41.730403    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xrkpf\""
Jul 06 18:24:41.738295 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:41.738268    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ghrjn"]
Jul 06 18:24:41.792791 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:41.792745    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56afe3f7-e3c1-4306-a07d-d8e2943ab2c6-cert\") pod \"odh-model-controller-7998c67dc4-ghrjn\" (UID: \"56afe3f7-e3c1-4306-a07d-d8e2943ab2c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-ghrjn"
Jul 06 18:24:41.792791 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:41.792789    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7bnv\" (UniqueName: \"kubernetes.io/projected/56afe3f7-e3c1-4306-a07d-d8e2943ab2c6-kube-api-access-z7bnv\") pod \"odh-model-controller-7998c67dc4-ghrjn\" (UID: \"56afe3f7-e3c1-4306-a07d-d8e2943ab2c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-ghrjn"
Jul 06 18:24:41.893959 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:41.893921    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56afe3f7-e3c1-4306-a07d-d8e2943ab2c6-cert\") pod \"odh-model-controller-7998c67dc4-ghrjn\" (UID: \"56afe3f7-e3c1-4306-a07d-d8e2943ab2c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-ghrjn"
Jul 06 18:24:41.893959 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:41.893968    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7bnv\" (UniqueName: \"kubernetes.io/projected/56afe3f7-e3c1-4306-a07d-d8e2943ab2c6-kube-api-access-z7bnv\") pod \"odh-model-controller-7998c67dc4-ghrjn\" (UID: \"56afe3f7-e3c1-4306-a07d-d8e2943ab2c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-ghrjn"
Jul 06 18:24:41.894243 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:24:41.894092    2581 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 06 18:24:41.894243 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:24:41.894158    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56afe3f7-e3c1-4306-a07d-d8e2943ab2c6-cert podName:56afe3f7-e3c1-4306-a07d-d8e2943ab2c6 nodeName:}" failed. No retries permitted until 2026-07-06 18:24:42.394140829 +0000 UTC m=+316.367590326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/56afe3f7-e3c1-4306-a07d-d8e2943ab2c6-cert") pod "odh-model-controller-7998c67dc4-ghrjn" (UID: "56afe3f7-e3c1-4306-a07d-d8e2943ab2c6") : secret "odh-model-controller-webhook-cert" not found
Jul 06 18:24:41.905619 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:41.905576    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7bnv\" (UniqueName: \"kubernetes.io/projected/56afe3f7-e3c1-4306-a07d-d8e2943ab2c6-kube-api-access-z7bnv\") pod \"odh-model-controller-7998c67dc4-ghrjn\" (UID: \"56afe3f7-e3c1-4306-a07d-d8e2943ab2c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-ghrjn"
Jul 06 18:24:42.378017 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.377977    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zzkcz"]
Jul 06 18:24:42.380101 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.380083    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-zzkcz"
Jul 06 18:24:42.383801 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.383779    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-hdcxt\""
Jul 06 18:24:42.383914 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.383887    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 06 18:24:42.398155 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.398119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56afe3f7-e3c1-4306-a07d-d8e2943ab2c6-cert\") pod \"odh-model-controller-7998c67dc4-ghrjn\" (UID: \"56afe3f7-e3c1-4306-a07d-d8e2943ab2c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-ghrjn"
Jul 06 18:24:42.398973 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.398947    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zzkcz"]
Jul 06 18:24:42.401366 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.401338    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56afe3f7-e3c1-4306-a07d-d8e2943ab2c6-cert\") pod \"odh-model-controller-7998c67dc4-ghrjn\" (UID: \"56afe3f7-e3c1-4306-a07d-d8e2943ab2c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-ghrjn"
Jul 06 18:24:42.499132 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.499090    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5774162-a816-4352-b3c7-01d7da9fc5e7-tls-certs\") pod \"model-serving-api-54d89bd5fc-zzkcz\" (UID: \"c5774162-a816-4352-b3c7-01d7da9fc5e7\") " pod="opendatahub/model-serving-api-54d89bd5fc-zzkcz"
Jul 06 18:24:42.499310 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.499139    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwn4l\" (UniqueName: \"kubernetes.io/projected/c5774162-a816-4352-b3c7-01d7da9fc5e7-kube-api-access-xwn4l\") pod \"model-serving-api-54d89bd5fc-zzkcz\" (UID: \"c5774162-a816-4352-b3c7-01d7da9fc5e7\") " pod="opendatahub/model-serving-api-54d89bd5fc-zzkcz"
Jul 06 18:24:42.600428 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.600385    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5774162-a816-4352-b3c7-01d7da9fc5e7-tls-certs\") pod \"model-serving-api-54d89bd5fc-zzkcz\" (UID: \"c5774162-a816-4352-b3c7-01d7da9fc5e7\") " pod="opendatahub/model-serving-api-54d89bd5fc-zzkcz"
Jul 06 18:24:42.600428 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.600427    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwn4l\" (UniqueName: \"kubernetes.io/projected/c5774162-a816-4352-b3c7-01d7da9fc5e7-kube-api-access-xwn4l\") pod \"model-serving-api-54d89bd5fc-zzkcz\" (UID: \"c5774162-a816-4352-b3c7-01d7da9fc5e7\") " pod="opendatahub/model-serving-api-54d89bd5fc-zzkcz"
Jul 06 18:24:42.602956 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.602932    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5774162-a816-4352-b3c7-01d7da9fc5e7-tls-certs\") pod \"model-serving-api-54d89bd5fc-zzkcz\" (UID: \"c5774162-a816-4352-b3c7-01d7da9fc5e7\") " pod="opendatahub/model-serving-api-54d89bd5fc-zzkcz"
Jul 06 18:24:42.608910 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.608882    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwn4l\" (UniqueName: \"kubernetes.io/projected/c5774162-a816-4352-b3c7-01d7da9fc5e7-kube-api-access-xwn4l\") pod \"model-serving-api-54d89bd5fc-zzkcz\" (UID: \"c5774162-a816-4352-b3c7-01d7da9fc5e7\") " pod="opendatahub/model-serving-api-54d89bd5fc-zzkcz"
Jul 06 18:24:42.639373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.639289    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ghrjn"
Jul 06 18:24:42.692962 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.692797    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-zzkcz"
Jul 06 18:24:42.778593 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.778563    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ghrjn"]
Jul 06 18:24:42.781721 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:24:42.781690    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56afe3f7_e3c1_4306_a07d_d8e2943ab2c6.slice/crio-1228c604d613d99f1c481d4f3f8989806c1f5f216062e7b480a7cdd59479f521 WatchSource:0}: Error finding container 1228c604d613d99f1c481d4f3f8989806c1f5f216062e7b480a7cdd59479f521: Status 404 returned error can't find the container with id 1228c604d613d99f1c481d4f3f8989806c1f5f216062e7b480a7cdd59479f521
Jul 06 18:24:42.783064 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.783047    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 18:24:42.839887 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:42.839851    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-zzkcz"]
Jul 06 18:24:42.842640 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:24:42.842618    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5774162_a816_4352_b3c7_01d7da9fc5e7.slice/crio-b062b1040b88bee3b0b2c35cc239c564db6cc990ac95b7e36abb7b0f6932f7a3 WatchSource:0}: Error finding container b062b1040b88bee3b0b2c35cc239c564db6cc990ac95b7e36abb7b0f6932f7a3: Status 404 returned error can't find the container with id b062b1040b88bee3b0b2c35cc239c564db6cc990ac95b7e36abb7b0f6932f7a3
Jul 06 18:24:43.676474 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:43.676418    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-zzkcz" event={"ID":"c5774162-a816-4352-b3c7-01d7da9fc5e7","Type":"ContainerStarted","Data":"b062b1040b88bee3b0b2c35cc239c564db6cc990ac95b7e36abb7b0f6932f7a3"}
Jul 06 18:24:43.678406 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:43.678360    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ghrjn" event={"ID":"56afe3f7-e3c1-4306-a07d-d8e2943ab2c6","Type":"ContainerStarted","Data":"1228c604d613d99f1c481d4f3f8989806c1f5f216062e7b480a7cdd59479f521"}
Jul 06 18:24:46.694557 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:46.694492    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ghrjn" event={"ID":"56afe3f7-e3c1-4306-a07d-d8e2943ab2c6","Type":"ContainerStarted","Data":"c55f65806dc3c0f83b3f72dfb6a934acde36ef61e1ab68840b68767c20634be3"}
Jul 06 18:24:46.695054 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:46.694714    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-ghrjn"
Jul 06 18:24:46.696662 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:46.696622    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-zzkcz" event={"ID":"c5774162-a816-4352-b3c7-01d7da9fc5e7","Type":"ContainerStarted","Data":"fa0c6d7ded3eacdbc6ee8de0fd413727f4ba675c6fa1680238f0beb914e9aaa2"}
Jul 06 18:24:46.696999 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:46.696973    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-zzkcz"
Jul 06 18:24:46.711143 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:46.711087    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-ghrjn" podStartSLOduration=2.312203144 podStartE2EDuration="5.711069168s" totalImagesPullingTime="3.398866024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:46.70994927 +0000 UTC m=+320.683398772" watchObservedRunningTime="2026-07-06 18:24:46.711069168 +0000 UTC m=+320.684518679"
Jul 06 18:24:46.726847 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:46.726787    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-zzkcz" podStartSLOduration=1.437789033 podStartE2EDuration="4.726762464s" totalImagesPullingTime="3.288973431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:46.726224335 +0000 UTC m=+320.699673873" watchObservedRunningTime="2026-07-06 18:24:46.726762464 +0000 UTC m=+320.700211977"
Jul 06 18:24:51.965207 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:51.965172    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-88568c598-wdxnb"]
Jul 06 18:24:51.968066 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:51.968042    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-88568c598-wdxnb"
Jul 06 18:24:51.976172 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:51.975928    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 06 18:24:51.976776 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:51.976711    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 18:24:51.976900 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:51.976795    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 18:24:51.977089 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:51.977073    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 06 18:24:51.986181 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:51.986144    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-88568c598-wdxnb"]
Jul 06 18:24:52.083190 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.083157    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de54f326-9c7e-4306-8d7c-d2c463dbed5f-tmp\") pod \"kube-auth-proxy-88568c598-wdxnb\" (UID: \"de54f326-9c7e-4306-8d7c-d2c463dbed5f\") " pod="openshift-ingress/kube-auth-proxy-88568c598-wdxnb"
Jul 06 18:24:52.083356 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.083208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6xvg\" (UniqueName: \"kubernetes.io/projected/de54f326-9c7e-4306-8d7c-d2c463dbed5f-kube-api-access-l6xvg\") pod \"kube-auth-proxy-88568c598-wdxnb\" (UID: \"de54f326-9c7e-4306-8d7c-d2c463dbed5f\") " pod="openshift-ingress/kube-auth-proxy-88568c598-wdxnb"
Jul 06 18:24:52.083356 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.083340    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de54f326-9c7e-4306-8d7c-d2c463dbed5f-tls-certs\") pod \"kube-auth-proxy-88568c598-wdxnb\" (UID: \"de54f326-9c7e-4306-8d7c-d2c463dbed5f\") " pod="openshift-ingress/kube-auth-proxy-88568c598-wdxnb"
Jul 06 18:24:52.115546 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.115504    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf"]
Jul 06 18:24:52.118663 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.118644    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf"
Jul 06 18:24:52.122975 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.122943    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-tcq5h\""
Jul 06 18:24:52.123113 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.122988    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 06 18:24:52.125660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.125619    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf"]
Jul 06 18:24:52.129346 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.129177    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf"
Jul 06 18:24:52.132399 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.132357    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 06 18:24:52.132679 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.132656    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-96pvs\""
Jul 06 18:24:52.133111 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.133089    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf"]
Jul 06 18:24:52.141980 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.141957    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf"]
Jul 06 18:24:52.184536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.184500    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgzgc\" (UniqueName: \"kubernetes.io/projected/69a9328b-5719-4081-ad43-de9724904356-kube-api-access-pgzgc\") pod \"llmisvc-controller-manager-777c6bd7d5-db4pf\" (UID: \"69a9328b-5719-4081-ad43-de9724904356\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf"
Jul 06 18:24:52.184536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.184538    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de54f326-9c7e-4306-8d7c-d2c463dbed5f-tmp\") pod \"kube-auth-proxy-88568c598-wdxnb\" (UID: \"de54f326-9c7e-4306-8d7c-d2c463dbed5f\") " pod="openshift-ingress/kube-auth-proxy-88568c598-wdxnb"
Jul 06 18:24:52.184821 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.184569    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6xvg\" (UniqueName: \"kubernetes.io/projected/de54f326-9c7e-4306-8d7c-d2c463dbed5f-kube-api-access-l6xvg\") pod \"kube-auth-proxy-88568c598-wdxnb\" (UID: \"de54f326-9c7e-4306-8d7c-d2c463dbed5f\") " pod="openshift-ingress/kube-auth-proxy-88568c598-wdxnb"
Jul 06 18:24:52.184821 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.184638    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0de3db7b-9518-43da-aa7c-31684a8bf481-cert\") pod \"kserve-controller-manager-d6f4b65b8-wtpwf\" (UID: \"0de3db7b-9518-43da-aa7c-31684a8bf481\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf"
Jul 06 18:24:52.184821 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.184699    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cqwf\" (UniqueName: \"kubernetes.io/projected/0de3db7b-9518-43da-aa7c-31684a8bf481-kube-api-access-2cqwf\") pod \"kserve-controller-manager-d6f4b65b8-wtpwf\" (UID: \"0de3db7b-9518-43da-aa7c-31684a8bf481\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf"
Jul 06 18:24:52.184821 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.184782    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69a9328b-5719-4081-ad43-de9724904356-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-db4pf\" (UID: \"69a9328b-5719-4081-ad43-de9724904356\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf"
Jul 06 18:24:52.185015 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.184854    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de54f326-9c7e-4306-8d7c-d2c463dbed5f-tls-certs\") pod \"kube-auth-proxy-88568c598-wdxnb\" (UID: \"de54f326-9c7e-4306-8d7c-d2c463dbed5f\") " pod="openshift-ingress/kube-auth-proxy-88568c598-wdxnb"
Jul 06 18:24:52.187080 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.187053    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de54f326-9c7e-4306-8d7c-d2c463dbed5f-tmp\") pod \"kube-auth-proxy-88568c598-wdxnb\" (UID: \"de54f326-9c7e-4306-8d7c-d2c463dbed5f\") " pod="openshift-ingress/kube-auth-proxy-88568c598-wdxnb"
Jul 06 18:24:52.187780 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.187754    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de54f326-9c7e-4306-8d7c-d2c463dbed5f-tls-certs\") pod \"kube-auth-proxy-88568c598-wdxnb\" (UID: \"de54f326-9c7e-4306-8d7c-d2c463dbed5f\") " pod="openshift-ingress/kube-auth-proxy-88568c598-wdxnb"
Jul 06 18:24:52.197072 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.197037    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6xvg\" (UniqueName: \"kubernetes.io/projected/de54f326-9c7e-4306-8d7c-d2c463dbed5f-kube-api-access-l6xvg\") pod \"kube-auth-proxy-88568c598-wdxnb\" (UID: \"de54f326-9c7e-4306-8d7c-d2c463dbed5f\") " pod="openshift-ingress/kube-auth-proxy-88568c598-wdxnb"
Jul 06 18:24:52.278848 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.278754    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-88568c598-wdxnb"
Jul 06 18:24:52.285449 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.285421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgzgc\" (UniqueName: \"kubernetes.io/projected/69a9328b-5719-4081-ad43-de9724904356-kube-api-access-pgzgc\") pod \"llmisvc-controller-manager-777c6bd7d5-db4pf\" (UID: \"69a9328b-5719-4081-ad43-de9724904356\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf"
Jul 06 18:24:52.285565 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.285477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0de3db7b-9518-43da-aa7c-31684a8bf481-cert\") pod \"kserve-controller-manager-d6f4b65b8-wtpwf\" (UID: \"0de3db7b-9518-43da-aa7c-31684a8bf481\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf"
Jul 06 18:24:52.285565 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.285503    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cqwf\" (UniqueName: \"kubernetes.io/projected/0de3db7b-9518-43da-aa7c-31684a8bf481-kube-api-access-2cqwf\") pod \"kserve-controller-manager-d6f4b65b8-wtpwf\" (UID: \"0de3db7b-9518-43da-aa7c-31684a8bf481\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf"
Jul 06 18:24:52.285565 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.285524    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69a9328b-5719-4081-ad43-de9724904356-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-db4pf\" (UID: \"69a9328b-5719-4081-ad43-de9724904356\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf"
Jul 06 18:24:52.285703 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:24:52.285638    2581 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 06 18:24:52.285770 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:24:52.285711    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0de3db7b-9518-43da-aa7c-31684a8bf481-cert podName:0de3db7b-9518-43da-aa7c-31684a8bf481 nodeName:}" failed. No retries permitted until 2026-07-06 18:24:52.785693215 +0000 UTC m=+326.759142725 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0de3db7b-9518-43da-aa7c-31684a8bf481-cert") pod "kserve-controller-manager-d6f4b65b8-wtpwf" (UID: "0de3db7b-9518-43da-aa7c-31684a8bf481") : secret "kserve-webhook-server-cert" not found
Jul 06 18:24:52.285770 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:24:52.285638    2581 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 06 18:24:52.285878 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:24:52.285786    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69a9328b-5719-4081-ad43-de9724904356-cert podName:69a9328b-5719-4081-ad43-de9724904356 nodeName:}" failed. No retries permitted until 2026-07-06 18:24:52.785773548 +0000 UTC m=+326.759223039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/69a9328b-5719-4081-ad43-de9724904356-cert") pod "llmisvc-controller-manager-777c6bd7d5-db4pf" (UID: "69a9328b-5719-4081-ad43-de9724904356") : secret "llmisvc-webhook-server-cert" not found
Jul 06 18:24:52.305162 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.305126    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgzgc\" (UniqueName: \"kubernetes.io/projected/69a9328b-5719-4081-ad43-de9724904356-kube-api-access-pgzgc\") pod \"llmisvc-controller-manager-777c6bd7d5-db4pf\" (UID: \"69a9328b-5719-4081-ad43-de9724904356\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf"
Jul 06 18:24:52.305300 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.305126    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cqwf\" (UniqueName: \"kubernetes.io/projected/0de3db7b-9518-43da-aa7c-31684a8bf481-kube-api-access-2cqwf\") pod \"kserve-controller-manager-d6f4b65b8-wtpwf\" (UID: \"0de3db7b-9518-43da-aa7c-31684a8bf481\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf"
Jul 06 18:24:52.433912 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.433888    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-88568c598-wdxnb"]
Jul 06 18:24:52.436471 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:24:52.436432    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde54f326_9c7e_4306_8d7c_d2c463dbed5f.slice/crio-ea9a273f69395c55ad1234da8bb02f4f046d16e128dbc78c7797207c5f42df76 WatchSource:0}: Error finding container ea9a273f69395c55ad1234da8bb02f4f046d16e128dbc78c7797207c5f42df76: Status 404 returned error can't find the container with id ea9a273f69395c55ad1234da8bb02f4f046d16e128dbc78c7797207c5f42df76
Jul 06 18:24:52.722600 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.722560    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-88568c598-wdxnb" event={"ID":"de54f326-9c7e-4306-8d7c-d2c463dbed5f","Type":"ContainerStarted","Data":"ea9a273f69395c55ad1234da8bb02f4f046d16e128dbc78c7797207c5f42df76"}
Jul 06 18:24:52.791194 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.791159    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0de3db7b-9518-43da-aa7c-31684a8bf481-cert\") pod \"kserve-controller-manager-d6f4b65b8-wtpwf\" (UID: \"0de3db7b-9518-43da-aa7c-31684a8bf481\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf"
Jul 06 18:24:52.791194 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.791199    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69a9328b-5719-4081-ad43-de9724904356-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-db4pf\" (UID: \"69a9328b-5719-4081-ad43-de9724904356\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf"
Jul 06 18:24:52.793643 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.793615    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0de3db7b-9518-43da-aa7c-31684a8bf481-cert\") pod \"kserve-controller-manager-d6f4b65b8-wtpwf\" (UID: \"0de3db7b-9518-43da-aa7c-31684a8bf481\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf"
Jul 06 18:24:52.793769 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:52.793656    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69a9328b-5719-4081-ad43-de9724904356-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-db4pf\" (UID: \"69a9328b-5719-4081-ad43-de9724904356\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf"
Jul 06 18:24:53.031700 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:53.031600    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf"
Jul 06 18:24:53.040573 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:53.040540    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf"
Jul 06 18:24:53.184992 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:53.184959    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf"]
Jul 06 18:24:53.189012 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:24:53.188943    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0de3db7b_9518_43da_aa7c_31684a8bf481.slice/crio-a991623e6e405d5a6de76e63fe5480ff894119d754ee780a5e500d8c844d48f4 WatchSource:0}: Error finding container a991623e6e405d5a6de76e63fe5480ff894119d754ee780a5e500d8c844d48f4: Status 404 returned error can't find the container with id a991623e6e405d5a6de76e63fe5480ff894119d754ee780a5e500d8c844d48f4
Jul 06 18:24:53.216098 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:53.216020    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf"]
Jul 06 18:24:53.218892 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:24:53.218853    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod69a9328b_5719_4081_ad43_de9724904356.slice/crio-17411c9de7b19c9a8025cffa9c485f55a9539b7236af0bbe0856fc0ba59de5c8 WatchSource:0}: Error finding container 17411c9de7b19c9a8025cffa9c485f55a9539b7236af0bbe0856fc0ba59de5c8: Status 404 returned error can't find the container with id 17411c9de7b19c9a8025cffa9c485f55a9539b7236af0bbe0856fc0ba59de5c8
Jul 06 18:24:53.728045 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:53.727997    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf" event={"ID":"0de3db7b-9518-43da-aa7c-31684a8bf481","Type":"ContainerStarted","Data":"a991623e6e405d5a6de76e63fe5480ff894119d754ee780a5e500d8c844d48f4"}
Jul 06 18:24:53.729341 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:53.729308    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf" event={"ID":"69a9328b-5719-4081-ad43-de9724904356","Type":"ContainerStarted","Data":"17411c9de7b19c9a8025cffa9c485f55a9539b7236af0bbe0856fc0ba59de5c8"}
Jul 06 18:24:54.142099 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.142062    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"]
Jul 06 18:24:54.148558 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.148527    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.151745 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.151712    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 06 18:24:54.151928 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.151787    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 06 18:24:54.151928 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.151799    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fsqrh\""
Jul 06 18:24:54.151928 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.151808    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 18:24:54.152110 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.152081    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 06 18:24:54.171393 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.171316    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"]
Jul 06 18:24:54.204677 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.204612    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/69977b49-a4e5-4f49-9b3d-efcd11d53c19-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.204879 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.204709    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/69977b49-a4e5-4f49-9b3d-efcd11d53c19-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.204879 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.204758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx62b\" (UniqueName: \"kubernetes.io/projected/69977b49-a4e5-4f49-9b3d-efcd11d53c19-kube-api-access-wx62b\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.204879 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.204803    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69977b49-a4e5-4f49-9b3d-efcd11d53c19-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.204879 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.204831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/69977b49-a4e5-4f49-9b3d-efcd11d53c19-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.205074 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.204896    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/69977b49-a4e5-4f49-9b3d-efcd11d53c19-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.205074 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.204920    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/69977b49-a4e5-4f49-9b3d-efcd11d53c19-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.306111 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.306072    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/69977b49-a4e5-4f49-9b3d-efcd11d53c19-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.306283 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.306125    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/69977b49-a4e5-4f49-9b3d-efcd11d53c19-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.306283 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.306166    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/69977b49-a4e5-4f49-9b3d-efcd11d53c19-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.306283 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.306211    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/69977b49-a4e5-4f49-9b3d-efcd11d53c19-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.306283 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.306239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx62b\" (UniqueName: \"kubernetes.io/projected/69977b49-a4e5-4f49-9b3d-efcd11d53c19-kube-api-access-wx62b\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.306283 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.306279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69977b49-a4e5-4f49-9b3d-efcd11d53c19-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.306549 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.306304    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/69977b49-a4e5-4f49-9b3d-efcd11d53c19-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.308006 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.307950    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/69977b49-a4e5-4f49-9b3d-efcd11d53c19-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.311027 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.310949    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/69977b49-a4e5-4f49-9b3d-efcd11d53c19-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.311537 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.311475    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/69977b49-a4e5-4f49-9b3d-efcd11d53c19-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.311629 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.311587    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/69977b49-a4e5-4f49-9b3d-efcd11d53c19-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.311841 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.311820    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/69977b49-a4e5-4f49-9b3d-efcd11d53c19-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.322560 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.322489    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69977b49-a4e5-4f49-9b3d-efcd11d53c19-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.323479 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.323421    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx62b\" (UniqueName: \"kubernetes.io/projected/69977b49-a4e5-4f49-9b3d-efcd11d53c19-kube-api-access-wx62b\") pod \"istiod-openshift-gateway-94bb8fbfd-h97l9\" (UID: \"69977b49-a4e5-4f49-9b3d-efcd11d53c19\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.460976 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.460897    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:54.813231 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:54.813153    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"]
Jul 06 18:24:55.396466 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:24:55.396427    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69977b49_a4e5_4f49_9b3d_efcd11d53c19.slice/crio-60917ec8fa37c09b9f6cf72f174c4a5084b2b933f14203a641d5d81b83f1d2b8 WatchSource:0}: Error finding container 60917ec8fa37c09b9f6cf72f174c4a5084b2b933f14203a641d5d81b83f1d2b8: Status 404 returned error can't find the container with id 60917ec8fa37c09b9f6cf72f174c4a5084b2b933f14203a641d5d81b83f1d2b8
Jul 06 18:24:55.742120 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:55.742031    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9" event={"ID":"69977b49-a4e5-4f49-9b3d-efcd11d53c19","Type":"ContainerStarted","Data":"60917ec8fa37c09b9f6cf72f174c4a5084b2b933f14203a641d5d81b83f1d2b8"}
Jul 06 18:24:55.743872 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:55.743835    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-88568c598-wdxnb" event={"ID":"de54f326-9c7e-4306-8d7c-d2c463dbed5f","Type":"ContainerStarted","Data":"2c35ab8798ab9327d800c4b10f565e2ddffcc15fe2591496b994789174734bfb"}
Jul 06 18:24:55.765281 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:55.765218    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-88568c598-wdxnb" podStartSLOduration=1.755625301 podStartE2EDuration="4.765194444s" totalImagesPullingTime="3.009569143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:55.763073153 +0000 UTC m=+329.736522665" watchObservedRunningTime="2026-07-06 18:24:55.765194444 +0000 UTC m=+329.738643954"
Jul 06 18:24:57.704231 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:57.704150    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-ghrjn"
Jul 06 18:24:57.706569 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:57.706544    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-zzkcz"
Jul 06 18:24:58.334065 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:58.334026    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 06 18:24:58.334156 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:58.334112    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 06 18:24:58.759674 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:58.759641    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf" event={"ID":"69a9328b-5719-4081-ad43-de9724904356","Type":"ContainerStarted","Data":"c8be204a479aa63250e3f96382ec9fa2c45c3e0a85e92f4f0425a932850a1661"}
Jul 06 18:24:58.760167 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:58.759728    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf"
Jul 06 18:24:58.761440 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:58.761410    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9" event={"ID":"69977b49-a4e5-4f49-9b3d-efcd11d53c19","Type":"ContainerStarted","Data":"9dd5aef83de0ababcfbc901513c6b04fed14f3cee73306d3199ca3c4fb6be63c"}
Jul 06 18:24:58.761583 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:58.761467    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:24:58.763070 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:58.763044    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf" event={"ID":"0de3db7b-9518-43da-aa7c-31684a8bf481","Type":"ContainerStarted","Data":"0a829f2963710dd31d3e130cae2947f0708a11c8b001c7ce746e3b384f88f3e2"}
Jul 06 18:24:58.763165 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:58.763098    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf"
Jul 06 18:24:58.777233 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:58.777184    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf" podStartSLOduration=1.726004764 podStartE2EDuration="6.77716843s" totalImagesPullingTime="5.051163666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:58.775822828 +0000 UTC m=+332.749272344" watchObservedRunningTime="2026-07-06 18:24:58.77716843 +0000 UTC m=+332.750617941"
Jul 06 18:24:58.792841 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:58.792785    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf" podStartSLOduration=1.703233118 podStartE2EDuration="6.792764152s" totalImagesPullingTime="5.089531034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:58.791931655 +0000 UTC m=+332.765381169" watchObservedRunningTime="2026-07-06 18:24:58.792764152 +0000 UTC m=+332.766213673"
Jul 06 18:24:58.811341 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:58.811286    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9" podStartSLOduration=1.876326371 podStartE2EDuration="4.811271245s" totalImagesPullingTime="2.934944874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:58.810036626 +0000 UTC m=+332.783486138" watchObservedRunningTime="2026-07-06 18:24:58.811271245 +0000 UTC m=+332.784720757"
Jul 06 18:24:59.769249 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:59.769213    2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-h97l9 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 18:24:59.769605 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:24:59.769276    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9" podUID="69977b49-a4e5-4f49-9b3d-efcd11d53c19" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 18:25:02.768691 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:25:02.768657    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-h97l9"
Jul 06 18:25:29.769603 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:25:29.769519    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-db4pf"
Jul 06 18:25:29.772651 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:25:29.772617    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wtpwf"
Jul 06 18:26:21.749771 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:21.749715    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dbp8j"]
Jul 06 18:26:21.753194 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:21.753175    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dbp8j"
Jul 06 18:26:21.755805 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:21.755780    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 18:26:21.756965 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:21.756949    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 06 18:26:21.757041 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:21.756964    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-wzrzt\""
Jul 06 18:26:21.757041 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:21.756965    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 18:26:21.773259 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:21.773226    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dbp8j"]
Jul 06 18:26:21.871794 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:21.871753    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lczdj\" (UniqueName: \"kubernetes.io/projected/da06585e-5e22-4597-bfb2-d2cbfbab9b16-kube-api-access-lczdj\") pod \"dns-operator-controller-manager-864dd78c4c-dbp8j\" (UID: \"da06585e-5e22-4597-bfb2-d2cbfbab9b16\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dbp8j"
Jul 06 18:26:21.972450 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:21.972410    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lczdj\" (UniqueName: \"kubernetes.io/projected/da06585e-5e22-4597-bfb2-d2cbfbab9b16-kube-api-access-lczdj\") pod \"dns-operator-controller-manager-864dd78c4c-dbp8j\" (UID: \"da06585e-5e22-4597-bfb2-d2cbfbab9b16\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dbp8j"
Jul 06 18:26:21.980982 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:21.980959    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lczdj\" (UniqueName: \"kubernetes.io/projected/da06585e-5e22-4597-bfb2-d2cbfbab9b16-kube-api-access-lczdj\") pod \"dns-operator-controller-manager-864dd78c4c-dbp8j\" (UID: \"da06585e-5e22-4597-bfb2-d2cbfbab9b16\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dbp8j"
Jul 06 18:26:22.064045 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:22.063952    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dbp8j"
Jul 06 18:26:22.198775 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:22.198728    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dbp8j"]
Jul 06 18:26:22.201940 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:26:22.201913    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda06585e_5e22_4597_bfb2_d2cbfbab9b16.slice/crio-57276447531aca06c051ff7f263748dd5927aab7eaa0e7d855bff0e8c8b8b81a WatchSource:0}: Error finding container 57276447531aca06c051ff7f263748dd5927aab7eaa0e7d855bff0e8c8b8b81a: Status 404 returned error can't find the container with id 57276447531aca06c051ff7f263748dd5927aab7eaa0e7d855bff0e8c8b8b81a
Jul 06 18:26:23.099932 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:23.099891    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dbp8j" event={"ID":"da06585e-5e22-4597-bfb2-d2cbfbab9b16","Type":"ContainerStarted","Data":"57276447531aca06c051ff7f263748dd5927aab7eaa0e7d855bff0e8c8b8b81a"}
Jul 06 18:26:23.744076 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:23.744045    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"]
Jul 06 18:26:23.747289 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:23.747267    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"
Jul 06 18:26:23.750030 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:23.750012    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-tdswx\""
Jul 06 18:26:23.756455 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:23.756433    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"]
Jul 06 18:26:23.791023 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:23.790974    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xpkd\" (UniqueName: \"kubernetes.io/projected/6767187e-be6c-4700-81a5-e6a2f9e35151-kube-api-access-4xpkd\") pod \"limitador-operator-controller-manager-859d7b78bd-mm76q\" (UID: \"6767187e-be6c-4700-81a5-e6a2f9e35151\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"
Jul 06 18:26:23.892431 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:23.892387    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xpkd\" (UniqueName: \"kubernetes.io/projected/6767187e-be6c-4700-81a5-e6a2f9e35151-kube-api-access-4xpkd\") pod \"limitador-operator-controller-manager-859d7b78bd-mm76q\" (UID: \"6767187e-be6c-4700-81a5-e6a2f9e35151\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"
Jul 06 18:26:23.901888 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:23.901855    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xpkd\" (UniqueName: \"kubernetes.io/projected/6767187e-be6c-4700-81a5-e6a2f9e35151-kube-api-access-4xpkd\") pod \"limitador-operator-controller-manager-859d7b78bd-mm76q\" (UID: \"6767187e-be6c-4700-81a5-e6a2f9e35151\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"
Jul 06 18:26:24.057774 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:24.057673    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"
Jul 06 18:26:24.313319 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:24.313265    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"]
Jul 06 18:26:24.316013 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:26:24.315988    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6767187e_be6c_4700_81a5_e6a2f9e35151.slice/crio-664f972f1f3a467594db8d55613c10800b1b72bb1ffe04b358c9c560014e6c1a WatchSource:0}: Error finding container 664f972f1f3a467594db8d55613c10800b1b72bb1ffe04b358c9c560014e6c1a: Status 404 returned error can't find the container with id 664f972f1f3a467594db8d55613c10800b1b72bb1ffe04b358c9c560014e6c1a
Jul 06 18:26:25.114554 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:25.114515    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q" event={"ID":"6767187e-be6c-4700-81a5-e6a2f9e35151","Type":"ContainerStarted","Data":"664f972f1f3a467594db8d55613c10800b1b72bb1ffe04b358c9c560014e6c1a"}
Jul 06 18:26:25.116158 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:25.116123    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dbp8j" event={"ID":"da06585e-5e22-4597-bfb2-d2cbfbab9b16","Type":"ContainerStarted","Data":"27447b61ffbef9cc84ae579530e07d1267bc9551b491bca1584f8f5a1585e3c6"}
Jul 06 18:26:25.116285 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:25.116258    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dbp8j"
Jul 06 18:26:25.136476 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:25.136422    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dbp8j" podStartSLOduration=2.103059046 podStartE2EDuration="4.136405804s" totalImagesPullingTime="2.033346758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:26:25.135207072 +0000 UTC m=+419.108656585" watchObservedRunningTime="2026-07-06 18:26:25.136405804 +0000 UTC m=+419.109855313"
Jul 06 18:26:27.126785 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:27.126708    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q" event={"ID":"6767187e-be6c-4700-81a5-e6a2f9e35151","Type":"ContainerStarted","Data":"20ab6ccc75f1da9510255d4b834bc6d28f40c4a0a4194f0065d7f5e29af8488d"}
Jul 06 18:26:27.127199 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:27.126856    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"
Jul 06 18:26:27.144000 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:27.143946    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q" podStartSLOduration=2.277090817 podStartE2EDuration="4.14392873s" totalImagesPullingTime="1.866837913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:26:27.142570043 +0000 UTC m=+421.116019558" watchObservedRunningTime="2026-07-06 18:26:27.14392873 +0000 UTC m=+421.117378244"
Jul 06 18:26:36.122930 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:36.122896    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dbp8j"
Jul 06 18:26:36.718257 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:36.718219    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"]
Jul 06 18:26:36.718501 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:36.718470    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q" podUID="6767187e-be6c-4700-81a5-e6a2f9e35151" containerName="manager" containerID="cri-o://20ab6ccc75f1da9510255d4b834bc6d28f40c4a0a4194f0065d7f5e29af8488d" gracePeriod=2
Jul 06 18:26:36.720503 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:36.720479    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"
Jul 06 18:26:36.745383 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:36.745342    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"]
Jul 06 18:26:36.760021 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:26:36.759980    2581 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6767187e-be6c-4700-81a5-e6a2f9e35151\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-mm76q\": pods \"limitador-operator-controller-manager-859d7b78bd-mm76q\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"
Jul 06 18:26:36.767812 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:26:36.767774    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mm76q\" is forbidden: User \"system:node:ip-10-0-141-229.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-229.ec2.internal' and this object" podUID="6767187e-be6c-4700-81a5-e6a2f9e35151" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"
Jul 06 18:26:36.962346 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:36.962316    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"
Jul 06 18:26:36.964594 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:26:36.964565    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mm76q\" is forbidden: User \"system:node:ip-10-0-141-229.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-229.ec2.internal' and this object" podUID="6767187e-be6c-4700-81a5-e6a2f9e35151" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"
Jul 06 18:26:37.104298 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:37.104198    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xpkd\" (UniqueName: \"kubernetes.io/projected/6767187e-be6c-4700-81a5-e6a2f9e35151-kube-api-access-4xpkd\") pod \"6767187e-be6c-4700-81a5-e6a2f9e35151\" (UID: \"6767187e-be6c-4700-81a5-e6a2f9e35151\") "
Jul 06 18:26:37.106370 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:37.106341    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6767187e-be6c-4700-81a5-e6a2f9e35151-kube-api-access-4xpkd" (OuterVolumeSpecName: "kube-api-access-4xpkd") pod "6767187e-be6c-4700-81a5-e6a2f9e35151" (UID: "6767187e-be6c-4700-81a5-e6a2f9e35151"). InnerVolumeSpecName "kube-api-access-4xpkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:26:37.171366 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:37.171329    2581 generic.go:356] "Generic (PLEG): container finished" podID="6767187e-be6c-4700-81a5-e6a2f9e35151" containerID="20ab6ccc75f1da9510255d4b834bc6d28f40c4a0a4194f0065d7f5e29af8488d" exitCode=0
Jul 06 18:26:37.171835 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:37.171424    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"
Jul 06 18:26:37.171835 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:37.171486    2581 scope.go:117] "RemoveContainer" containerID="20ab6ccc75f1da9510255d4b834bc6d28f40c4a0a4194f0065d7f5e29af8488d"
Jul 06 18:26:37.174428 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:26:37.174390    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mm76q\" is forbidden: User \"system:node:ip-10-0-141-229.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-229.ec2.internal' and this object" podUID="6767187e-be6c-4700-81a5-e6a2f9e35151" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"
Jul 06 18:26:37.180373 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:37.180354    2581 scope.go:117] "RemoveContainer" containerID="20ab6ccc75f1da9510255d4b834bc6d28f40c4a0a4194f0065d7f5e29af8488d"
Jul 06 18:26:37.180696 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:26:37.180666    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20ab6ccc75f1da9510255d4b834bc6d28f40c4a0a4194f0065d7f5e29af8488d\": container with ID starting with 20ab6ccc75f1da9510255d4b834bc6d28f40c4a0a4194f0065d7f5e29af8488d not found: ID does not exist" containerID="20ab6ccc75f1da9510255d4b834bc6d28f40c4a0a4194f0065d7f5e29af8488d"
Jul 06 18:26:37.180781 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:37.180706    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20ab6ccc75f1da9510255d4b834bc6d28f40c4a0a4194f0065d7f5e29af8488d"} err="failed to get container status \"20ab6ccc75f1da9510255d4b834bc6d28f40c4a0a4194f0065d7f5e29af8488d\": rpc error: code = NotFound desc = could not find container \"20ab6ccc75f1da9510255d4b834bc6d28f40c4a0a4194f0065d7f5e29af8488d\": container with ID starting with 20ab6ccc75f1da9510255d4b834bc6d28f40c4a0a4194f0065d7f5e29af8488d not found: ID does not exist"
Jul 06 18:26:37.182720 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:26:37.182694    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mm76q\" is forbidden: User \"system:node:ip-10-0-141-229.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-229.ec2.internal' and this object" podUID="6767187e-be6c-4700-81a5-e6a2f9e35151" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"
Jul 06 18:26:37.205144 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:37.205105    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xpkd\" (UniqueName: \"kubernetes.io/projected/6767187e-be6c-4700-81a5-e6a2f9e35151-kube-api-access-4xpkd\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:26:37.469804 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:26:37.469763    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mm76q\" is forbidden: User \"system:node:ip-10-0-141-229.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-229.ec2.internal' and this object" podUID="6767187e-be6c-4700-81a5-e6a2f9e35151" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mm76q"
Jul 06 18:26:37.470902 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:37.470881    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6767187e-be6c-4700-81a5-e6a2f9e35151" path="/var/lib/kubelet/pods/6767187e-be6c-4700-81a5-e6a2f9e35151/volumes"
Jul 06 18:26:38.026566 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:38.026531    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"]
Jul 06 18:26:38.026935 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:38.026914    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6767187e-be6c-4700-81a5-e6a2f9e35151" containerName="manager"
Jul 06 18:26:38.026935 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:38.026934    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6767187e-be6c-4700-81a5-e6a2f9e35151" containerName="manager"
Jul 06 18:26:38.027157 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:38.027004    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6767187e-be6c-4700-81a5-e6a2f9e35151" containerName="manager"
Jul 06 18:26:38.031255 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:38.031233    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"
Jul 06 18:26:38.034514 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:38.034490    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9gb28\""
Jul 06 18:26:38.046948 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:38.046918    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"]
Jul 06 18:26:38.113042 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:38.113007    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/30a008aa-7524-4307-ac5a-c2a647236139-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w9cfx\" (UID: \"30a008aa-7524-4307-ac5a-c2a647236139\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"
Jul 06 18:26:38.113228 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:38.113063    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pbv6\" (UniqueName: \"kubernetes.io/projected/30a008aa-7524-4307-ac5a-c2a647236139-kube-api-access-2pbv6\") pod \"kuadrant-operator-controller-manager-86878c769f-w9cfx\" (UID: \"30a008aa-7524-4307-ac5a-c2a647236139\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"
Jul 06 18:26:38.214148 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:38.214108    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/30a008aa-7524-4307-ac5a-c2a647236139-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w9cfx\" (UID: \"30a008aa-7524-4307-ac5a-c2a647236139\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"
Jul 06 18:26:38.214552 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:38.214172    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pbv6\" (UniqueName: \"kubernetes.io/projected/30a008aa-7524-4307-ac5a-c2a647236139-kube-api-access-2pbv6\") pod \"kuadrant-operator-controller-manager-86878c769f-w9cfx\" (UID: \"30a008aa-7524-4307-ac5a-c2a647236139\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"
Jul 06 18:26:38.214552 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:38.214517    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/30a008aa-7524-4307-ac5a-c2a647236139-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w9cfx\" (UID: \"30a008aa-7524-4307-ac5a-c2a647236139\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"
Jul 06 18:26:38.230762 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:38.230691    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pbv6\" (UniqueName: \"kubernetes.io/projected/30a008aa-7524-4307-ac5a-c2a647236139-kube-api-access-2pbv6\") pod \"kuadrant-operator-controller-manager-86878c769f-w9cfx\" (UID: \"30a008aa-7524-4307-ac5a-c2a647236139\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"
Jul 06 18:26:38.341157 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:38.341041    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"
Jul 06 18:26:38.477633 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:38.477594    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"]
Jul 06 18:26:38.481355 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:26:38.481324    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30a008aa_7524_4307_ac5a_c2a647236139.slice/crio-43e2de04343f0c319a7c9602096c7477330939c043bc6162f9992a0b022e1317 WatchSource:0}: Error finding container 43e2de04343f0c319a7c9602096c7477330939c043bc6162f9992a0b022e1317: Status 404 returned error can't find the container with id 43e2de04343f0c319a7c9602096c7477330939c043bc6162f9992a0b022e1317
Jul 06 18:26:39.181881 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:39.181848    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx" event={"ID":"30a008aa-7524-4307-ac5a-c2a647236139","Type":"ContainerStarted","Data":"43e2de04343f0c319a7c9602096c7477330939c043bc6162f9992a0b022e1317"}
Jul 06 18:26:43.202504 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:43.202402    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx" event={"ID":"30a008aa-7524-4307-ac5a-c2a647236139","Type":"ContainerStarted","Data":"4f3a035e49e6e0ede7e8ae61385194e2353490f7b743d8a70c1ec6f513bc2589"}
Jul 06 18:26:43.202504 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:43.202492    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"
Jul 06 18:26:43.223212 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:43.223157    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx" podStartSLOduration=0.809635091 podStartE2EDuration="5.2231392s" totalImagesPullingTime="4.413504109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:26:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:26:43.222049119 +0000 UTC m=+437.195498632" watchObservedRunningTime="2026-07-06 18:26:43.2231392 +0000 UTC m=+437.196588708"
Jul 06 18:26:54.208703 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:26:54.208619    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"
Jul 06 18:27:13.665053 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:13.665008    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:27:13.671571 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:13.671545    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"
Jul 06 18:27:13.674870 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:13.674846    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 06 18:27:13.675665 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:13.675648    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jqz8q\""
Jul 06 18:27:13.683990 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:13.683964    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:27:13.693774 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:13.693744    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn2fz\" (UniqueName: \"kubernetes.io/projected/76a5fa6a-1679-49b2-a189-7a80b64d7248-kube-api-access-sn2fz\") pod \"limitador-limitador-54d6d988d5-d8lz6\" (UID: \"76a5fa6a-1679-49b2-a189-7a80b64d7248\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"
Jul 06 18:27:13.693893 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:13.693866    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/76a5fa6a-1679-49b2-a189-7a80b64d7248-config-file\") pod \"limitador-limitador-54d6d988d5-d8lz6\" (UID: \"76a5fa6a-1679-49b2-a189-7a80b64d7248\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"
Jul 06 18:27:13.752944 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:13.752907    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:27:13.794575 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:13.794534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn2fz\" (UniqueName: \"kubernetes.io/projected/76a5fa6a-1679-49b2-a189-7a80b64d7248-kube-api-access-sn2fz\") pod \"limitador-limitador-54d6d988d5-d8lz6\" (UID: \"76a5fa6a-1679-49b2-a189-7a80b64d7248\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"
Jul 06 18:27:13.794771 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:13.794622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/76a5fa6a-1679-49b2-a189-7a80b64d7248-config-file\") pod \"limitador-limitador-54d6d988d5-d8lz6\" (UID: \"76a5fa6a-1679-49b2-a189-7a80b64d7248\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"
Jul 06 18:27:13.795290 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:13.795270    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/76a5fa6a-1679-49b2-a189-7a80b64d7248-config-file\") pod \"limitador-limitador-54d6d988d5-d8lz6\" (UID: \"76a5fa6a-1679-49b2-a189-7a80b64d7248\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"
Jul 06 18:27:13.805673 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:13.805639    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn2fz\" (UniqueName: \"kubernetes.io/projected/76a5fa6a-1679-49b2-a189-7a80b64d7248-kube-api-access-sn2fz\") pod \"limitador-limitador-54d6d988d5-d8lz6\" (UID: \"76a5fa6a-1679-49b2-a189-7a80b64d7248\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"
Jul 06 18:27:13.981430 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:13.981327    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"
Jul 06 18:27:14.112879 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:14.112845    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:27:14.116080 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:27:14.116034    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76a5fa6a_1679_49b2_a189_7a80b64d7248.slice/crio-f4c2f9138b1dd9b68a5c65b4e68eb8963887502edeefc976749864f0299541e3 WatchSource:0}: Error finding container f4c2f9138b1dd9b68a5c65b4e68eb8963887502edeefc976749864f0299541e3: Status 404 returned error can't find the container with id f4c2f9138b1dd9b68a5c65b4e68eb8963887502edeefc976749864f0299541e3
Jul 06 18:27:14.339354 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:14.339276    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-d8lz6" event={"ID":"76a5fa6a-1679-49b2-a189-7a80b64d7248","Type":"ContainerStarted","Data":"f4c2f9138b1dd9b68a5c65b4e68eb8963887502edeefc976749864f0299541e3"}
Jul 06 18:27:17.357664 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:17.357627    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-d8lz6" event={"ID":"76a5fa6a-1679-49b2-a189-7a80b64d7248","Type":"ContainerStarted","Data":"b4e886d2297a71058a4e1a7ab744717cc923e02386bd2939eca5da42c3baf65e"}
Jul 06 18:27:17.358119 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:17.357765    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"
Jul 06 18:27:17.373459 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:17.373383    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-d8lz6" podStartSLOduration=1.7897182919999999 podStartE2EDuration="4.373367101s" totalImagesPullingTime="2.583648809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:27:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:27:17.372911044 +0000 UTC m=+471.346360556" watchObservedRunningTime="2026-07-06 18:27:17.373367101 +0000 UTC m=+471.346816650"
Jul 06 18:27:28.362928 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:27:28.362896    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"
Jul 06 18:29:27.389702 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:27.389674    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:29:27.390467 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:27.389917    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:29:29.810722 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:29.810677    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-xgbrn"]
Jul 06 18:29:29.814214 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:29.814182    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-xgbrn"
Jul 06 18:29:29.817125 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:29.817109    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 06 18:29:29.825970 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:29.825941    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-xgbrn"]
Jul 06 18:29:29.843084 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:29.843052    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:29:29.904559 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:29.904518    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffc659f1-d433-4a93-83f5-dc4601f49a6f-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-xgbrn\" (UID: \"ffc659f1-d433-4a93-83f5-dc4601f49a6f\") " pod="openshift-ingress/payload-processing-7bbcb654f4-xgbrn"
Jul 06 18:29:29.904758 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:29.904666    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt4rq\" (UniqueName: \"kubernetes.io/projected/ffc659f1-d433-4a93-83f5-dc4601f49a6f-kube-api-access-wt4rq\") pod \"payload-processing-7bbcb654f4-xgbrn\" (UID: \"ffc659f1-d433-4a93-83f5-dc4601f49a6f\") " pod="openshift-ingress/payload-processing-7bbcb654f4-xgbrn"
Jul 06 18:29:30.005658 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:30.005618    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffc659f1-d433-4a93-83f5-dc4601f49a6f-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-xgbrn\" (UID: \"ffc659f1-d433-4a93-83f5-dc4601f49a6f\") " pod="openshift-ingress/payload-processing-7bbcb654f4-xgbrn"
Jul 06 18:29:30.005658 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:30.005654    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt4rq\" (UniqueName: \"kubernetes.io/projected/ffc659f1-d433-4a93-83f5-dc4601f49a6f-kube-api-access-wt4rq\") pod \"payload-processing-7bbcb654f4-xgbrn\" (UID: \"ffc659f1-d433-4a93-83f5-dc4601f49a6f\") " pod="openshift-ingress/payload-processing-7bbcb654f4-xgbrn"
Jul 06 18:29:30.006344 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:30.006322    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffc659f1-d433-4a93-83f5-dc4601f49a6f-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-xgbrn\" (UID: \"ffc659f1-d433-4a93-83f5-dc4601f49a6f\") " pod="openshift-ingress/payload-processing-7bbcb654f4-xgbrn"
Jul 06 18:29:30.016321 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:30.016285    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt4rq\" (UniqueName: \"kubernetes.io/projected/ffc659f1-d433-4a93-83f5-dc4601f49a6f-kube-api-access-wt4rq\") pod \"payload-processing-7bbcb654f4-xgbrn\" (UID: \"ffc659f1-d433-4a93-83f5-dc4601f49a6f\") " pod="openshift-ingress/payload-processing-7bbcb654f4-xgbrn"
Jul 06 18:29:30.124214 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:30.124176    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-xgbrn"
Jul 06 18:29:30.261790 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:30.261753    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-xgbrn"]
Jul 06 18:29:30.264981 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:29:30.264950    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc659f1_d433_4a93_83f5_dc4601f49a6f.slice/crio-df09ea5ba5091a1ad15e37343fef35102de863a3648fe0d2b2e115b5f94e839d WatchSource:0}: Error finding container df09ea5ba5091a1ad15e37343fef35102de863a3648fe0d2b2e115b5f94e839d: Status 404 returned error can't find the container with id df09ea5ba5091a1ad15e37343fef35102de863a3648fe0d2b2e115b5f94e839d
Jul 06 18:29:30.941050 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:30.941012    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-xgbrn" event={"ID":"ffc659f1-d433-4a93-83f5-dc4601f49a6f","Type":"ContainerStarted","Data":"df09ea5ba5091a1ad15e37343fef35102de863a3648fe0d2b2e115b5f94e839d"}
Jul 06 18:29:33.959235 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:33.959196    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-xgbrn" event={"ID":"ffc659f1-d433-4a93-83f5-dc4601f49a6f","Type":"ContainerStarted","Data":"5c1a6e1557cea4dc0925b014cc58329dc1d66dfd9d8273b801ecae899e1155ad"}
Jul 06 18:29:33.959638 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:33.959337    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-xgbrn"
Jul 06 18:29:33.977465 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:33.977408    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bbcb654f4-xgbrn" podStartSLOduration=1.755847672 podStartE2EDuration="4.977390893s" totalImagesPullingTime="3.221543221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:29:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:29:33.976377168 +0000 UTC m=+607.949826682" watchObservedRunningTime="2026-07-06 18:29:33.977390893 +0000 UTC m=+607.950840427"
Jul 06 18:29:44.965558 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:29:44.965527    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-xgbrn"
Jul 06 18:30:00.145552 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:00.145461    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722710-zptrv"]
Jul 06 18:30:00.152406 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:00.152382    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722710-zptrv"
Jul 06 18:30:00.155854 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:00.155832    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-hgx44\""
Jul 06 18:30:00.160140 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:00.160113    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722710-zptrv"]
Jul 06 18:30:00.261695 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:00.261652    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sqww\" (UniqueName: \"kubernetes.io/projected/c7b32498-7ed8-4ab1-b35b-3f7af4ea37de-kube-api-access-7sqww\") pod \"maas-api-key-cleanup-29722710-zptrv\" (UID: \"c7b32498-7ed8-4ab1-b35b-3f7af4ea37de\") " pod="opendatahub/maas-api-key-cleanup-29722710-zptrv"
Jul 06 18:30:00.363086 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:00.363035    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sqww\" (UniqueName: \"kubernetes.io/projected/c7b32498-7ed8-4ab1-b35b-3f7af4ea37de-kube-api-access-7sqww\") pod \"maas-api-key-cleanup-29722710-zptrv\" (UID: \"c7b32498-7ed8-4ab1-b35b-3f7af4ea37de\") " pod="opendatahub/maas-api-key-cleanup-29722710-zptrv"
Jul 06 18:30:00.378404 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:00.378367    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sqww\" (UniqueName: \"kubernetes.io/projected/c7b32498-7ed8-4ab1-b35b-3f7af4ea37de-kube-api-access-7sqww\") pod \"maas-api-key-cleanup-29722710-zptrv\" (UID: \"c7b32498-7ed8-4ab1-b35b-3f7af4ea37de\") " pod="opendatahub/maas-api-key-cleanup-29722710-zptrv"
Jul 06 18:30:00.463913 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:00.463814    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722710-zptrv"
Jul 06 18:30:00.598943 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:00.598894    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722710-zptrv"]
Jul 06 18:30:00.601151 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:30:00.601122    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7b32498_7ed8_4ab1_b35b_3f7af4ea37de.slice/crio-98a1a2b0e3280cb2cfaff272742b15fd35efa3a0c53f5fcb40e20aee8e3fc3b8 WatchSource:0}: Error finding container 98a1a2b0e3280cb2cfaff272742b15fd35efa3a0c53f5fcb40e20aee8e3fc3b8: Status 404 returned error can't find the container with id 98a1a2b0e3280cb2cfaff272742b15fd35efa3a0c53f5fcb40e20aee8e3fc3b8
Jul 06 18:30:00.603225 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:00.603204    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 18:30:01.077018 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:01.076981    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722710-zptrv" event={"ID":"c7b32498-7ed8-4ab1-b35b-3f7af4ea37de","Type":"ContainerStarted","Data":"98a1a2b0e3280cb2cfaff272742b15fd35efa3a0c53f5fcb40e20aee8e3fc3b8"}
Jul 06 18:30:04.092510 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:04.092419    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722710-zptrv" event={"ID":"c7b32498-7ed8-4ab1-b35b-3f7af4ea37de","Type":"ContainerStarted","Data":"c4b43ad2a2fb7a497671146b70ae6a74974954da11ca77f7705f814e3351c01a"}
Jul 06 18:30:04.109059 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:04.109008    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722710-zptrv" podStartSLOduration=1.298442691 podStartE2EDuration="4.108993628s" totalImagesPullingTime="2.810550937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:30:04.107960749 +0000 UTC m=+638.081410264" watchObservedRunningTime="2026-07-06 18:30:04.108993628 +0000 UTC m=+638.082443141"
Jul 06 18:30:05.098130 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:05.098091    2581 generic.go:356] "Generic (PLEG): container finished" podID="c7b32498-7ed8-4ab1-b35b-3f7af4ea37de" containerID="c4b43ad2a2fb7a497671146b70ae6a74974954da11ca77f7705f814e3351c01a" exitCode=0
Jul 06 18:30:05.098526 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:05.098179    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722710-zptrv" event={"ID":"c7b32498-7ed8-4ab1-b35b-3f7af4ea37de","Type":"ContainerDied","Data":"c4b43ad2a2fb7a497671146b70ae6a74974954da11ca77f7705f814e3351c01a"}
Jul 06 18:30:06.235820 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:06.235793    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722710-zptrv"
Jul 06 18:30:06.313250 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:06.313208    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sqww\" (UniqueName: \"kubernetes.io/projected/c7b32498-7ed8-4ab1-b35b-3f7af4ea37de-kube-api-access-7sqww\") pod \"c7b32498-7ed8-4ab1-b35b-3f7af4ea37de\" (UID: \"c7b32498-7ed8-4ab1-b35b-3f7af4ea37de\") "
Jul 06 18:30:06.315431 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:06.315397    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7b32498-7ed8-4ab1-b35b-3f7af4ea37de-kube-api-access-7sqww" (OuterVolumeSpecName: "kube-api-access-7sqww") pod "c7b32498-7ed8-4ab1-b35b-3f7af4ea37de" (UID: "c7b32498-7ed8-4ab1-b35b-3f7af4ea37de"). InnerVolumeSpecName "kube-api-access-7sqww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:30:06.414506 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:06.414461    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sqww\" (UniqueName: \"kubernetes.io/projected/c7b32498-7ed8-4ab1-b35b-3f7af4ea37de-kube-api-access-7sqww\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:30:07.108949 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:07.108914    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722710-zptrv" event={"ID":"c7b32498-7ed8-4ab1-b35b-3f7af4ea37de","Type":"ContainerDied","Data":"98a1a2b0e3280cb2cfaff272742b15fd35efa3a0c53f5fcb40e20aee8e3fc3b8"}
Jul 06 18:30:07.109136 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:07.108964    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="98a1a2b0e3280cb2cfaff272742b15fd35efa3a0c53f5fcb40e20aee8e3fc3b8"
Jul 06 18:30:07.109136 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:07.108923    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722710-zptrv"
Jul 06 18:30:23.451392 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.451359    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"]
Jul 06 18:30:23.451804 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.451719    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7b32498-7ed8-4ab1-b35b-3f7af4ea37de" containerName="cleanup"
Jul 06 18:30:23.451804 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.451747    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7b32498-7ed8-4ab1-b35b-3f7af4ea37de" containerName="cleanup"
Jul 06 18:30:23.451892 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.451822    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7b32498-7ed8-4ab1-b35b-3f7af4ea37de" containerName="cleanup"
Jul 06 18:30:23.454456 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.454439    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.458504 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.458480    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-86kpz\""
Jul 06 18:30:23.458964 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.458944    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 06 18:30:23.459340 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.459322    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 06 18:30:23.460331 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.460313    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 06 18:30:23.475335 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.475306    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"]
Jul 06 18:30:23.554938 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.554900    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.555114 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.554943    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.555114 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.554975    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8hw5\" (UniqueName: \"kubernetes.io/projected/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-kube-api-access-w8hw5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.555114 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.555071    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.555114 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.555106    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.555269 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.555185    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.656296 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.656250    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.656296 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.656298    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.656560 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.656363    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.656560 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.656410    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.656560 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.656431    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.656560 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.656460    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8hw5\" (UniqueName: \"kubernetes.io/projected/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-kube-api-access-w8hw5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.656902 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.656868    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.657007 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.656873    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.657007 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.656942    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.658834 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.658801    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.659331 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.659306    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.667135 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.667098    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8hw5\" (UniqueName: \"kubernetes.io/projected/fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c-kube-api-access-w8hw5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz\" (UID: \"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.770809 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.770697    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:23.915555 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:23.915516    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"]
Jul 06 18:30:23.918717 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:30:23.918690    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcbd6d15_7dcc_46d0_92b5_62fd6f887d4c.slice/crio-19f678d4877b7f79e3bcad304a62a33d3bdfac141ed67fd20ccebd6858c14e6b WatchSource:0}: Error finding container 19f678d4877b7f79e3bcad304a62a33d3bdfac141ed67fd20ccebd6858c14e6b: Status 404 returned error can't find the container with id 19f678d4877b7f79e3bcad304a62a33d3bdfac141ed67fd20ccebd6858c14e6b
Jul 06 18:30:24.038565 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:24.038480    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:30:24.185104 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:24.185066    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz" event={"ID":"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c","Type":"ContainerStarted","Data":"19f678d4877b7f79e3bcad304a62a33d3bdfac141ed67fd20ccebd6858c14e6b"}
Jul 06 18:30:25.193044 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:25.192999    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz" event={"ID":"fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c","Type":"ContainerStarted","Data":"e1c9b1067462ada7282b2f146968968c26abb5bd5db95ddced90832916d0705b"}
Jul 06 18:30:25.212415 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:25.212354    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz" podStartSLOduration=1.075767081 podStartE2EDuration="2.212338274s" totalImagesPullingTime="1.136571193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:30:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:30:25.211017715 +0000 UTC m=+659.184467267" watchObservedRunningTime="2026-07-06 18:30:25.212338274 +0000 UTC m=+659.185787787"
Jul 06 18:30:31.044675 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:31.044636    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:30:33.771612 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:33.771573    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:33.772014 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:33.771643    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:33.784310 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:33.784289    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:34.246129 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:34.246094    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz"
Jul 06 18:30:35.448975 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:35.448934    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:30:52.543414 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:52.543376    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:30:59.035021 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.034986    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"]
Jul 06 18:30:59.040228 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.040204    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.042976 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.042946    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 06 18:30:59.051093 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.051066    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"]
Jul 06 18:30:59.151684 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.151648    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2968477-c746-42f4-baf0-a3a82444982f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.151903 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.151710    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnrd7\" (UniqueName: \"kubernetes.io/projected/f2968477-c746-42f4-baf0-a3a82444982f-kube-api-access-bnrd7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.151903 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.151764    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2968477-c746-42f4-baf0-a3a82444982f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.151903 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.151787    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2968477-c746-42f4-baf0-a3a82444982f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.151903 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.151830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2968477-c746-42f4-baf0-a3a82444982f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.152054 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.151896    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2968477-c746-42f4-baf0-a3a82444982f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.253105 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.253069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2968477-c746-42f4-baf0-a3a82444982f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.253301 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.253119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnrd7\" (UniqueName: \"kubernetes.io/projected/f2968477-c746-42f4-baf0-a3a82444982f-kube-api-access-bnrd7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.253301 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.253138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2968477-c746-42f4-baf0-a3a82444982f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.253301 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.253251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2968477-c746-42f4-baf0-a3a82444982f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.253457 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.253321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2968477-c746-42f4-baf0-a3a82444982f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.253457 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.253366    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2968477-c746-42f4-baf0-a3a82444982f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.253535 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.253492    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2968477-c746-42f4-baf0-a3a82444982f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.253609 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.253590    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2968477-c746-42f4-baf0-a3a82444982f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.253643 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.253623    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2968477-c746-42f4-baf0-a3a82444982f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.255270 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.255250    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2968477-c746-42f4-baf0-a3a82444982f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.255615 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.255598    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2968477-c746-42f4-baf0-a3a82444982f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.261011 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.260991    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnrd7\" (UniqueName: \"kubernetes.io/projected/f2968477-c746-42f4-baf0-a3a82444982f-kube-api-access-bnrd7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-qc74q\" (UID: \"f2968477-c746-42f4-baf0-a3a82444982f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.356655 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.356553    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:30:59.487774 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:30:59.487751    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"]
Jul 06 18:30:59.490995 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:30:59.490967    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2968477_c746_42f4_baf0_a3a82444982f.slice/crio-ccfda0b460daf780607f815f9dfdf1d5d7fdd418b469ac673217891d9abe33b5 WatchSource:0}: Error finding container ccfda0b460daf780607f815f9dfdf1d5d7fdd418b469ac673217891d9abe33b5: Status 404 returned error can't find the container with id ccfda0b460daf780607f815f9dfdf1d5d7fdd418b469ac673217891d9abe33b5
Jul 06 18:31:00.351030 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:31:00.350991    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q" event={"ID":"f2968477-c746-42f4-baf0-a3a82444982f","Type":"ContainerStarted","Data":"ede5f1de2bec1b9712f6e2a04d3e7c018a55ba9f4b8e62e7ae6552dd726dea3c"}
Jul 06 18:31:00.351546 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:31:00.351054    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q" event={"ID":"f2968477-c746-42f4-baf0-a3a82444982f","Type":"ContainerStarted","Data":"ccfda0b460daf780607f815f9dfdf1d5d7fdd418b469ac673217891d9abe33b5"}
Jul 06 18:31:00.370242 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:31:00.370183    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q" podStartSLOduration=1.162242844 podStartE2EDuration="1.370163481s" totalImagesPullingTime="207.920637ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:30:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:31:00.369757307 +0000 UTC m=+694.343206810" watchObservedRunningTime="2026-07-06 18:31:00.370163481 +0000 UTC m=+694.343612994"
Jul 06 18:31:01.343709 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:31:01.343673    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:31:09.357237 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:31:09.357190    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:31:09.357671 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:31:09.357268    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:31:09.369610 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:31:09.369586    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:31:09.405793 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:31:09.405755    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-qc74q"
Jul 06 18:31:15.846540 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:31:15.846498    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:32:41.849597 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:32:41.849551    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:32:52.340071 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:32:52.339984    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:33:03.440627 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:33:03.440593    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:33:12.141655 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:33:12.141611    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:33:23.342078 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:33:23.342045    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:34:27.416119 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:34:27.416021    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:34:27.416942 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:34:27.416919    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:34:29.639242 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:34:29.639204    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:34:38.640090 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:34:38.640055    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:34:47.638222 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:34:47.638183    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:34:53.458283 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:34:53.458236    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:35:04.246906 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:35:04.246869    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:35:16.136180 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:35:16.136147    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:35:27.190411 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:35:27.190368    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:35:37.040362 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:35:37.040325    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:35:59.437188 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:35:59.437152    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:36:08.662476 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:36:08.662436    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:36:32.240250 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:36:32.240206    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:36:42.242689 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:36:42.242644    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:37:01.838839 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:37:01.838803    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:37:11.038996 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:37:11.038958    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:37:20.451474 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:37:20.451386    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:37:30.436656 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:37:30.436618    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:37:38.939662 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:37:38.939625    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:38:06.639674 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:06.639635    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"]
Jul 06 18:38:06.640325 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:06.639899    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx" podUID="30a008aa-7524-4307-ac5a-c2a647236139" containerName="manager" containerID="cri-o://4f3a035e49e6e0ede7e8ae61385194e2353490f7b743d8a70c1ec6f513bc2589" gracePeriod=10
Jul 06 18:38:10.197009 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.196983    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"
Jul 06 18:38:10.239655 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.239569    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pbv6\" (UniqueName: \"kubernetes.io/projected/30a008aa-7524-4307-ac5a-c2a647236139-kube-api-access-2pbv6\") pod \"30a008aa-7524-4307-ac5a-c2a647236139\" (UID: \"30a008aa-7524-4307-ac5a-c2a647236139\") "
Jul 06 18:38:10.239815 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.239679    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/30a008aa-7524-4307-ac5a-c2a647236139-extensions-socket-volume\") pod \"30a008aa-7524-4307-ac5a-c2a647236139\" (UID: \"30a008aa-7524-4307-ac5a-c2a647236139\") "
Jul 06 18:38:10.240072 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.240051    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30a008aa-7524-4307-ac5a-c2a647236139-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "30a008aa-7524-4307-ac5a-c2a647236139" (UID: "30a008aa-7524-4307-ac5a-c2a647236139"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:38:10.241571 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.241548    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30a008aa-7524-4307-ac5a-c2a647236139-kube-api-access-2pbv6" (OuterVolumeSpecName: "kube-api-access-2pbv6") pod "30a008aa-7524-4307-ac5a-c2a647236139" (UID: "30a008aa-7524-4307-ac5a-c2a647236139"). InnerVolumeSpecName "kube-api-access-2pbv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:38:10.278353 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.278322    2581 generic.go:356] "Generic (PLEG): container finished" podID="30a008aa-7524-4307-ac5a-c2a647236139" containerID="4f3a035e49e6e0ede7e8ae61385194e2353490f7b743d8a70c1ec6f513bc2589" exitCode=0
Jul 06 18:38:10.278544 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.278365    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"
Jul 06 18:38:10.278544 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.278406    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx" event={"ID":"30a008aa-7524-4307-ac5a-c2a647236139","Type":"ContainerDied","Data":"4f3a035e49e6e0ede7e8ae61385194e2353490f7b743d8a70c1ec6f513bc2589"}
Jul 06 18:38:10.278544 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.278471    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx" event={"ID":"30a008aa-7524-4307-ac5a-c2a647236139","Type":"ContainerDied","Data":"43e2de04343f0c319a7c9602096c7477330939c043bc6162f9992a0b022e1317"}
Jul 06 18:38:10.278544 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.278502    2581 scope.go:117] "RemoveContainer" containerID="4f3a035e49e6e0ede7e8ae61385194e2353490f7b743d8a70c1ec6f513bc2589"
Jul 06 18:38:10.287793 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.287775    2581 scope.go:117] "RemoveContainer" containerID="4f3a035e49e6e0ede7e8ae61385194e2353490f7b743d8a70c1ec6f513bc2589"
Jul 06 18:38:10.288071 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:38:10.288052    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f3a035e49e6e0ede7e8ae61385194e2353490f7b743d8a70c1ec6f513bc2589\": container with ID starting with 4f3a035e49e6e0ede7e8ae61385194e2353490f7b743d8a70c1ec6f513bc2589 not found: ID does not exist" containerID="4f3a035e49e6e0ede7e8ae61385194e2353490f7b743d8a70c1ec6f513bc2589"
Jul 06 18:38:10.288108 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.288080    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f3a035e49e6e0ede7e8ae61385194e2353490f7b743d8a70c1ec6f513bc2589"} err="failed to get container status \"4f3a035e49e6e0ede7e8ae61385194e2353490f7b743d8a70c1ec6f513bc2589\": rpc error: code = NotFound desc = could not find container \"4f3a035e49e6e0ede7e8ae61385194e2353490f7b743d8a70c1ec6f513bc2589\": container with ID starting with 4f3a035e49e6e0ede7e8ae61385194e2353490f7b743d8a70c1ec6f513bc2589 not found: ID does not exist"
Jul 06 18:38:10.301283 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.301249    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"]
Jul 06 18:38:10.304941 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.304916    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w9cfx"]
Jul 06 18:38:10.341128 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.341071    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pbv6\" (UniqueName: \"kubernetes.io/projected/30a008aa-7524-4307-ac5a-c2a647236139-kube-api-access-2pbv6\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:38:10.341128 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:10.341119    2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/30a008aa-7524-4307-ac5a-c2a647236139-extensions-socket-volume\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:38:11.471596 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:38:11.471557    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30a008aa-7524-4307-ac5a-c2a647236139" path="/var/lib/kubelet/pods/30a008aa-7524-4307-ac5a-c2a647236139/volumes"
Jul 06 18:39:07.915574 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:07.915472    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk"]
Jul 06 18:39:07.916206 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:07.915997    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30a008aa-7524-4307-ac5a-c2a647236139" containerName="manager"
Jul 06 18:39:07.916206 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:07.916019    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="30a008aa-7524-4307-ac5a-c2a647236139" containerName="manager"
Jul 06 18:39:07.916206 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:07.916107    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="30a008aa-7524-4307-ac5a-c2a647236139" containerName="manager"
Jul 06 18:39:07.919127 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:07.919104    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk"
Jul 06 18:39:07.921976 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:07.921955    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9gb28\""
Jul 06 18:39:07.931772 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:07.931722    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk"]
Jul 06 18:39:08.026405 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:08.026352    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwgxt\" (UniqueName: \"kubernetes.io/projected/a49361a1-af70-4f8e-96a1-8bb0165c0dfd-kube-api-access-dwgxt\") pod \"kuadrant-operator-controller-manager-86878c769f-gwwxk\" (UID: \"a49361a1-af70-4f8e-96a1-8bb0165c0dfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk"
Jul 06 18:39:08.026600 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:08.026490    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a49361a1-af70-4f8e-96a1-8bb0165c0dfd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gwwxk\" (UID: \"a49361a1-af70-4f8e-96a1-8bb0165c0dfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk"
Jul 06 18:39:08.127070 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:08.127030    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a49361a1-af70-4f8e-96a1-8bb0165c0dfd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gwwxk\" (UID: \"a49361a1-af70-4f8e-96a1-8bb0165c0dfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk"
Jul 06 18:39:08.127239 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:08.127080    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwgxt\" (UniqueName: \"kubernetes.io/projected/a49361a1-af70-4f8e-96a1-8bb0165c0dfd-kube-api-access-dwgxt\") pod \"kuadrant-operator-controller-manager-86878c769f-gwwxk\" (UID: \"a49361a1-af70-4f8e-96a1-8bb0165c0dfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk"
Jul 06 18:39:08.127507 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:08.127482    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a49361a1-af70-4f8e-96a1-8bb0165c0dfd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gwwxk\" (UID: \"a49361a1-af70-4f8e-96a1-8bb0165c0dfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk"
Jul 06 18:39:08.135873 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:08.135840    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwgxt\" (UniqueName: \"kubernetes.io/projected/a49361a1-af70-4f8e-96a1-8bb0165c0dfd-kube-api-access-dwgxt\") pod \"kuadrant-operator-controller-manager-86878c769f-gwwxk\" (UID: \"a49361a1-af70-4f8e-96a1-8bb0165c0dfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk"
Jul 06 18:39:08.230209 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:08.230114    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk"
Jul 06 18:39:08.378225 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:08.378193    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk"]
Jul 06 18:39:08.381722 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:39:08.381691    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda49361a1_af70_4f8e_96a1_8bb0165c0dfd.slice/crio-579bbd87645621281fe0a9b5adab96f733470e1cdc62c19237f2a8181fbbe49e WatchSource:0}: Error finding container 579bbd87645621281fe0a9b5adab96f733470e1cdc62c19237f2a8181fbbe49e: Status 404 returned error can't find the container with id 579bbd87645621281fe0a9b5adab96f733470e1cdc62c19237f2a8181fbbe49e
Jul 06 18:39:08.540945 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:08.540843    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk" event={"ID":"a49361a1-af70-4f8e-96a1-8bb0165c0dfd","Type":"ContainerStarted","Data":"fd35795cba2109dca980b3cd85088164836a4e80c23e4f00414a05da40b4dbf2"}
Jul 06 18:39:08.540945 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:08.540907    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk" event={"ID":"a49361a1-af70-4f8e-96a1-8bb0165c0dfd","Type":"ContainerStarted","Data":"579bbd87645621281fe0a9b5adab96f733470e1cdc62c19237f2a8181fbbe49e"}
Jul 06 18:39:08.541135 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:08.540954    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk"
Jul 06 18:39:08.561410 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:08.561360    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk" podStartSLOduration=1.561336486 podStartE2EDuration="1.561336486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:39:08.56086993 +0000 UTC m=+1182.534319442" watchObservedRunningTime="2026-07-06 18:39:08.561336486 +0000 UTC m=+1182.534786000"
Jul 06 18:39:19.547503 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:19.547467    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gwwxk"
Jul 06 18:39:27.440955 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:27.440923    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:39:27.443537 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:27.443502    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:39:29.747663 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:29.747628    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:39:41.243445 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:41.243402    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:39:50.247997 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:39:50.247960    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:40:02.541583 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:40:02.541544    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:40:13.733536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:40:13.733499    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:40:24.540456 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:40:24.540360    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:40:33.641789 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:40:33.641754    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:40:44.736265 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:40:44.736227    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:40:55.045071 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:40:55.045036    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:41:04.638484 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:41:04.638446    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:41:15.970597 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:41:15.970560    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:41:39.646945 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:41:39.646907    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:41:47.285058 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:41:47.285000    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:41:57.054896 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:41:57.054807    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:42:08.442941 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:42:08.442904    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:42:25.334449 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:42:25.334413    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:42:34.144420 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:42:34.144377    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:42:43.440350 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:42:43.440310    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:42:51.338053 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:42:51.338005    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:43:02.151541 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:43:02.151506    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:43:11.543036 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:43:11.542997    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:43:24.738639 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:43:24.738598    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:43:34.443248 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:43:34.443210    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:43:44.039225 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:43:44.039190    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:43:53.940923 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:43:53.940887    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:44:06.948564 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:44:06.948528    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:44:16.350185 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:44:16.350146    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:44:27.466665 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:44:27.466632    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:44:27.475961 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:44:27.475928    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:44:27.789948 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:44:27.789855    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:44:39.243585 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:44:39.243539    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:44:48.337639 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:44:48.337554    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:44:56.747230 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:44:56.747194    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:45:09.344037 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:45:09.344003    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:45:18.245568 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:45:18.245532    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:45:26.747214 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:45:26.747174    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:45:34.434766 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:45:34.434709    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:45:45.749645 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:45:45.749608    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:45:56.536256 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:45:56.536219    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:47:42.570874 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:47:42.570836    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:47:50.692985 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:47:50.692894    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:48:00.354774 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:00.354722    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:48:10.254181 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:10.254147    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:48:17.558185 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:17.558146    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:48:50.833161 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:50.833123    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:48:58.745666 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.745626    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"]
Jul 06 18:48:58.749029 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.749005    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.751977 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.751955    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f8683f4a\"/\"e2e-lifecycle-model-f8683f4a-kserve-self-signed-certs\""
Jul 06 18:48:58.753343 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.753321    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f8683f4a\"/\"kube-root-ca.crt\""
Jul 06 18:48:58.753343 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.753340    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f8683f4a\"/\"openshift-service-ca.crt\""
Jul 06 18:48:58.753497 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.753386    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f8683f4a\"/\"default-dockercfg-tqhpq\""
Jul 06 18:48:58.757231 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.757206    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"]
Jul 06 18:48:58.830811 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.830771    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-tls-certs\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.830981 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.830817    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qssj\" (UniqueName: \"kubernetes.io/projected/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-kube-api-access-6qssj\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.830981 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.830917    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-tmp-dir\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.830981 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.830957    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-model-cache\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.831089 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.831050    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-dshm\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.831089 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.831084    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-home\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.931981 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.931933    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-dshm\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.932171 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.931989    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-home\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.932171 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.932047    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-tls-certs\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.932171 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.932073    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qssj\" (UniqueName: \"kubernetes.io/projected/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-kube-api-access-6qssj\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.932171 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.932118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-tmp-dir\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.932171 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.932153    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-model-cache\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.932514 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.932486    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-home\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.932604 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.932533    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-model-cache\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.932815 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.932795    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-tmp-dir\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.934843 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.934818    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-dshm\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.935016 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.934999    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-tls-certs\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:58.941896 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:58.941871    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qssj\" (UniqueName: \"kubernetes.io/projected/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-kube-api-access-6qssj\") pod \"e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") " pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:59.059052 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:59.058957    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:48:59.193998 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:48:59.193960    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"]
Jul 06 18:48:59.198262 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:48:59.198223    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf90c4871_73fb_47f1_ad04_48f1f9e1a72c.slice/crio-93f8a0b4c0a1d66d6e7ac7c17558a32f9d6604b7fa628d76129fbe9b05e79085 WatchSource:0}: Error finding container 93f8a0b4c0a1d66d6e7ac7c17558a32f9d6604b7fa628d76129fbe9b05e79085: Status 404 returned error can't find the container with id 93f8a0b4c0a1d66d6e7ac7c17558a32f9d6604b7fa628d76129fbe9b05e79085
Jul 06 18:49:00.204574 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:00.204532    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn" event={"ID":"f90c4871-73fb-47f1-ad04-48f1f9e1a72c","Type":"ContainerStarted","Data":"d87211de3de8815e76f1e92568f8d04b2e5c58fa7d1f9a648d4d6388b0dd9752"}
Jul 06 18:49:00.205028 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:00.204585    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn" event={"ID":"f90c4871-73fb-47f1-ad04-48f1f9e1a72c","Type":"ContainerStarted","Data":"93f8a0b4c0a1d66d6e7ac7c17558a32f9d6604b7fa628d76129fbe9b05e79085"}
Jul 06 18:49:00.220805 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:00.220714    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn" podStartSLOduration=2.220693497 podStartE2EDuration="2.220693497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:48:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:49:00.220428484 +0000 UTC m=+1774.193877999" watchObservedRunningTime="2026-07-06 18:49:00.220693497 +0000 UTC m=+1774.194143012"
Jul 06 18:49:03.035245 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:03.035206    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:49:09.060124 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:09.060083    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:49:09.060531 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:09.060169    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:49:09.072886 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:09.072855    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:49:09.256563 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:09.256532    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:49:26.251066 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:26.251028    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:49:27.506966 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:27.506934    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:49:27.509561 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:27.509536    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:49:36.647592 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:36.647558    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:49:42.193475 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:42.193437    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"]
Jul 06 18:49:46.634878 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:46.634839    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:49:57.214387 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.214325    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn" podUID="f90c4871-73fb-47f1-ad04-48f1f9e1a72c" containerName="main" containerID="cri-o://d87211de3de8815e76f1e92568f8d04b2e5c58fa7d1f9a648d4d6388b0dd9752" gracePeriod=45
Jul 06 18:49:57.463917 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.463887    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:49:57.472230 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.472159    2581 generic.go:356] "Generic (PLEG): container finished" podID="f90c4871-73fb-47f1-ad04-48f1f9e1a72c" containerID="d87211de3de8815e76f1e92568f8d04b2e5c58fa7d1f9a648d4d6388b0dd9752" exitCode=0
Jul 06 18:49:57.472360 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.472254    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"
Jul 06 18:49:57.472930 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.472908    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn" event={"ID":"f90c4871-73fb-47f1-ad04-48f1f9e1a72c","Type":"ContainerDied","Data":"d87211de3de8815e76f1e92568f8d04b2e5c58fa7d1f9a648d4d6388b0dd9752"}
Jul 06 18:49:57.473021 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.472947    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn" event={"ID":"f90c4871-73fb-47f1-ad04-48f1f9e1a72c","Type":"ContainerDied","Data":"93f8a0b4c0a1d66d6e7ac7c17558a32f9d6604b7fa628d76129fbe9b05e79085"}
Jul 06 18:49:57.473021 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.472971    2581 scope.go:117] "RemoveContainer" containerID="d87211de3de8815e76f1e92568f8d04b2e5c58fa7d1f9a648d4d6388b0dd9752"
Jul 06 18:49:57.481590 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.481563    2581 scope.go:117] "RemoveContainer" containerID="d87211de3de8815e76f1e92568f8d04b2e5c58fa7d1f9a648d4d6388b0dd9752"
Jul 06 18:49:57.482067 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:49:57.482040    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d87211de3de8815e76f1e92568f8d04b2e5c58fa7d1f9a648d4d6388b0dd9752\": container with ID starting with d87211de3de8815e76f1e92568f8d04b2e5c58fa7d1f9a648d4d6388b0dd9752 not found: ID does not exist" containerID="d87211de3de8815e76f1e92568f8d04b2e5c58fa7d1f9a648d4d6388b0dd9752"
Jul 06 18:49:57.482195 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.482074    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d87211de3de8815e76f1e92568f8d04b2e5c58fa7d1f9a648d4d6388b0dd9752"} err="failed to get container status \"d87211de3de8815e76f1e92568f8d04b2e5c58fa7d1f9a648d4d6388b0dd9752\": rpc error: code = NotFound desc = could not find container \"d87211de3de8815e76f1e92568f8d04b2e5c58fa7d1f9a648d4d6388b0dd9752\": container with ID starting with d87211de3de8815e76f1e92568f8d04b2e5c58fa7d1f9a648d4d6388b0dd9752 not found: ID does not exist"
Jul 06 18:49:57.559163 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.559119    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qssj\" (UniqueName: \"kubernetes.io/projected/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-kube-api-access-6qssj\") pod \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") "
Jul 06 18:49:57.559347 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.559186    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-model-cache\") pod \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") "
Jul 06 18:49:57.559347 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.559217    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-tls-certs\") pod \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") "
Jul 06 18:49:57.559347 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.559329    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-tmp-dir\") pod \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") "
Jul 06 18:49:57.559502 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.559378    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-home\") pod \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") "
Jul 06 18:49:57.559502 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.559435    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-dshm\") pod \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\" (UID: \"f90c4871-73fb-47f1-ad04-48f1f9e1a72c\") "
Jul 06 18:49:57.559502 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.559486    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-model-cache" (OuterVolumeSpecName: "model-cache") pod "f90c4871-73fb-47f1-ad04-48f1f9e1a72c" (UID: "f90c4871-73fb-47f1-ad04-48f1f9e1a72c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:49:57.559663 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.559610    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f90c4871-73fb-47f1-ad04-48f1f9e1a72c" (UID: "f90c4871-73fb-47f1-ad04-48f1f9e1a72c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:49:57.559717 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.559684    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-home" (OuterVolumeSpecName: "home") pod "f90c4871-73fb-47f1-ad04-48f1f9e1a72c" (UID: "f90c4871-73fb-47f1-ad04-48f1f9e1a72c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:49:57.559797 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.559725    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-model-cache\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:49:57.559797 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.559778    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-tmp-dir\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:49:57.561420 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.561392    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f90c4871-73fb-47f1-ad04-48f1f9e1a72c" (UID: "f90c4871-73fb-47f1-ad04-48f1f9e1a72c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:49:57.561536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.561440    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-kube-api-access-6qssj" (OuterVolumeSpecName: "kube-api-access-6qssj") pod "f90c4871-73fb-47f1-ad04-48f1f9e1a72c" (UID: "f90c4871-73fb-47f1-ad04-48f1f9e1a72c"). InnerVolumeSpecName "kube-api-access-6qssj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:49:57.561536 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.561439    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-dshm" (OuterVolumeSpecName: "dshm") pod "f90c4871-73fb-47f1-ad04-48f1f9e1a72c" (UID: "f90c4871-73fb-47f1-ad04-48f1f9e1a72c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:49:57.661035 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.660996    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qssj\" (UniqueName: \"kubernetes.io/projected/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-kube-api-access-6qssj\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:49:57.661035 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.661030    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-tls-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:49:57.661035 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.661040    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-home\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:49:57.661282 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.661048    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f90c4871-73fb-47f1-ad04-48f1f9e1a72c-dshm\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:49:57.794409 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.794365    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"]
Jul 06 18:49:57.798285 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:57.798254    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-f8683f4a/e2e-lifecycle-model-f8683f4a-kserve-67c9cf5775-652zn"]
Jul 06 18:49:59.471576 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:49:59.471541    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f90c4871-73fb-47f1-ad04-48f1f9e1a72c" path="/var/lib/kubelet/pods/f90c4871-73fb-47f1-ad04-48f1f9e1a72c/volumes"
Jul 06 18:50:21.645814 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:50:21.645770    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:51:31.442557 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:31.442515    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:51:42.649545 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:42.649511    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:51:52.029112 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.029078    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"]
Jul 06 18:51:52.029501 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.029438    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f90c4871-73fb-47f1-ad04-48f1f9e1a72c" containerName="main"
Jul 06 18:51:52.029501 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.029451    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f90c4871-73fb-47f1-ad04-48f1f9e1a72c" containerName="main"
Jul 06 18:51:52.029573 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.029514    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f90c4871-73fb-47f1-ad04-48f1f9e1a72c" containerName="main"
Jul 06 18:51:52.032259 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.032240    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.036095 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.036071    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-f2348c\"/\"kube-root-ca.crt\""
Jul 06 18:51:52.036226 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.036210    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-f2348c\"/\"test-model-f2348c-kserve-self-signed-certs\""
Jul 06 18:51:52.036295 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.036243    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-f2348c\"/\"default-dockercfg-9b49j\""
Jul 06 18:51:52.036295 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.036244    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-f2348c\"/\"openshift-service-ca.crt\""
Jul 06 18:51:52.040047 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.040025    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"]
Jul 06 18:51:52.135463 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.135430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l877\" (UniqueName: \"kubernetes.io/projected/f2278735-e384-4164-b664-cdd911a018f4-kube-api-access-8l877\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.135656 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.135470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-home\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.135656 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.135568    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-dshm\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.135656 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.135636    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-tmp-dir\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.135813 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.135673    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-model-cache\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.135813 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.135689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2278735-e384-4164-b664-cdd911a018f4-tls-certs\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.236551 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.236518    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l877\" (UniqueName: \"kubernetes.io/projected/f2278735-e384-4164-b664-cdd911a018f4-kube-api-access-8l877\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.237106 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.236556    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-home\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.237266 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.237182    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-dshm\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.237326 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.237273    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-tmp-dir\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.237326 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.237316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-model-cache\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.237454 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.237423    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-home\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.237670 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.237646    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-model-cache\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.237772 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.237673    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-tmp-dir\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.237772 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.237764    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2278735-e384-4164-b664-cdd911a018f4-tls-certs\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.241717 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.240926    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2278735-e384-4164-b664-cdd911a018f4-tls-certs\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.244315 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.244287    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-dshm\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.246151 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.246130    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l877\" (UniqueName: \"kubernetes.io/projected/f2278735-e384-4164-b664-cdd911a018f4-kube-api-access-8l877\") pod \"test-model-f2348c-kserve-6b586fcbf-6rcjn\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") " pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.341624 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.341529    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:51:52.475660 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.475637    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"]
Jul 06 18:51:52.478149 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:51:52.478122    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2278735_e384_4164_b664_cdd911a018f4.slice/crio-96d3fb83b441526d9ee13434f0317190d9b6c196ef940993607a3e58b23d59d0 WatchSource:0}: Error finding container 96d3fb83b441526d9ee13434f0317190d9b6c196ef940993607a3e58b23d59d0: Status 404 returned error can't find the container with id 96d3fb83b441526d9ee13434f0317190d9b6c196ef940993607a3e58b23d59d0
Jul 06 18:51:52.992165 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.992126    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn" event={"ID":"f2278735-e384-4164-b664-cdd911a018f4","Type":"ContainerStarted","Data":"5170a1970f7e68e22bf78137cbe7f36d6e675a06860271f81f3a5d07d913f77a"}
Jul 06 18:51:52.992345 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:52.992177    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn" event={"ID":"f2278735-e384-4164-b664-cdd911a018f4","Type":"ContainerStarted","Data":"96d3fb83b441526d9ee13434f0317190d9b6c196ef940993607a3e58b23d59d0"}
Jul 06 18:51:53.014618 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:53.014570    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn" podStartSLOduration=1.014550039 podStartE2EDuration="1.014550039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:51:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:51:53.013370164 +0000 UTC m=+1946.986819666" watchObservedRunningTime="2026-07-06 18:51:53.014550039 +0000 UTC m=+1946.987999552"
Jul 06 18:51:55.753594 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:51:55.753556    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:52:02.342076 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:02.342028    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:52:02.342493 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:02.342094    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:52:02.354115 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:02.354086    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:52:03.049595 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:03.049562    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:52:09.662645 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:09.662610    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:52:21.356353 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.356265    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"]
Jul 06 18:52:21.360237 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.360214    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.363110 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.363084    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ef5ef7\"/\"openshift-service-ca.crt\""
Jul 06 18:52:21.363245 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.363148    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ef5ef7\"/\"default-dockercfg-lrzps\""
Jul 06 18:52:21.363245 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.363156    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ef5ef7\"/\"kube-root-ca.crt\""
Jul 06 18:52:21.363245 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.363183    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ef5ef7\"/\"test-model-ef5ef7-kserve-self-signed-certs\""
Jul 06 18:52:21.368105 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.367761    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"]
Jul 06 18:52:21.387322 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.387290    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6207f293-aceb-425c-bff0-84abc5d646e5-tls-certs\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.387322 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.387322    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-home\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.387494 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.387348    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-dshm\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.387494 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.387364    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-tmp-dir\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.387494 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.387449    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqxfg\" (UniqueName: \"kubernetes.io/projected/6207f293-aceb-425c-bff0-84abc5d646e5-kube-api-access-bqxfg\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.387494 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.387482    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-model-cache\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.490843 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.490799    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqxfg\" (UniqueName: \"kubernetes.io/projected/6207f293-aceb-425c-bff0-84abc5d646e5-kube-api-access-bqxfg\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.491041 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.490856    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-model-cache\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.491237 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.491217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6207f293-aceb-425c-bff0-84abc5d646e5-tls-certs\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.491296 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.491258    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-home\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.491393 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.491379    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-dshm\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.491445 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.491411    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-tmp-dir\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.491751 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.491704    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-model-cache\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.493376 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.492727    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-home\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.493376 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.493021    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-tmp-dir\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.496039 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.495034    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-dshm\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.497714 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.496050    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6207f293-aceb-425c-bff0-84abc5d646e5-tls-certs\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.505891 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.505870    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqxfg\" (UniqueName: \"kubernetes.io/projected/6207f293-aceb-425c-bff0-84abc5d646e5-kube-api-access-bqxfg\") pod \"test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") " pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.670414 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.670376    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:21.807377 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:21.807350    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"]
Jul 06 18:52:21.809549 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:52:21.809520    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6207f293_aceb_425c_bff0_84abc5d646e5.slice/crio-63ebadc7364c6b866638902f944dac46579c5a547384390a43bedcfa3e92f0ea WatchSource:0}: Error finding container 63ebadc7364c6b866638902f944dac46579c5a547384390a43bedcfa3e92f0ea: Status 404 returned error can't find the container with id 63ebadc7364c6b866638902f944dac46579c5a547384390a43bedcfa3e92f0ea
Jul 06 18:52:22.124880 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:22.124840    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z" event={"ID":"6207f293-aceb-425c-bff0-84abc5d646e5","Type":"ContainerStarted","Data":"01471944513d11ceb054672d578e2af505c61501e967362eccfbde21fc5edf81"}
Jul 06 18:52:22.124880 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:22.124883    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z" event={"ID":"6207f293-aceb-425c-bff0-84abc5d646e5","Type":"ContainerStarted","Data":"63ebadc7364c6b866638902f944dac46579c5a547384390a43bedcfa3e92f0ea"}
Jul 06 18:52:22.145591 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:22.145541    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z" podStartSLOduration=1.145526044 podStartE2EDuration="1.145526044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:52:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:52:22.143367713 +0000 UTC m=+1976.116817224" watchObservedRunningTime="2026-07-06 18:52:22.145526044 +0000 UTC m=+1976.118975555"
Jul 06 18:52:26.962698 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:26.962663    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:52:31.670758 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:31.670692    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:31.671194 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:31.670785    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:31.683110 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:31.683081    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:32.182044 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:32.182003    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:52:55.618428 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:55.618393    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"]
Jul 06 18:52:56.591848 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:52:56.591808    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"]
Jul 06 18:53:00.347103 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:00.347067    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:53:10.637841 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:10.637799    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn" podUID="f2278735-e384-4164-b664-cdd911a018f4" containerName="main" containerID="cri-o://5170a1970f7e68e22bf78137cbe7f36d6e675a06860271f81f3a5d07d913f77a" gracePeriod=45
Jul 06 18:53:10.886071 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:10.886045    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:53:10.942675 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:10.942588    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2278735-e384-4164-b664-cdd911a018f4-tls-certs\") pod \"f2278735-e384-4164-b664-cdd911a018f4\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") "
Jul 06 18:53:10.942675 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:10.942648    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l877\" (UniqueName: \"kubernetes.io/projected/f2278735-e384-4164-b664-cdd911a018f4-kube-api-access-8l877\") pod \"f2278735-e384-4164-b664-cdd911a018f4\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") "
Jul 06 18:53:10.942675 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:10.942668    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-home\") pod \"f2278735-e384-4164-b664-cdd911a018f4\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") "
Jul 06 18:53:10.942940 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:10.942757    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-tmp-dir\") pod \"f2278735-e384-4164-b664-cdd911a018f4\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") "
Jul 06 18:53:10.942940 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:10.942780    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-model-cache\") pod \"f2278735-e384-4164-b664-cdd911a018f4\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") "
Jul 06 18:53:10.942940 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:10.942811    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-dshm\") pod \"f2278735-e384-4164-b664-cdd911a018f4\" (UID: \"f2278735-e384-4164-b664-cdd911a018f4\") "
Jul 06 18:53:10.943056 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:10.943022    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-home" (OuterVolumeSpecName: "home") pod "f2278735-e384-4164-b664-cdd911a018f4" (UID: "f2278735-e384-4164-b664-cdd911a018f4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:53:10.943056 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:10.943037    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f2278735-e384-4164-b664-cdd911a018f4" (UID: "f2278735-e384-4164-b664-cdd911a018f4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:53:10.943127 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:10.943064    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-model-cache" (OuterVolumeSpecName: "model-cache") pod "f2278735-e384-4164-b664-cdd911a018f4" (UID: "f2278735-e384-4164-b664-cdd911a018f4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:53:10.945002 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:10.944970    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2278735-e384-4164-b664-cdd911a018f4-kube-api-access-8l877" (OuterVolumeSpecName: "kube-api-access-8l877") pod "f2278735-e384-4164-b664-cdd911a018f4" (UID: "f2278735-e384-4164-b664-cdd911a018f4"). InnerVolumeSpecName "kube-api-access-8l877". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:53:10.945002 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:10.944971    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2278735-e384-4164-b664-cdd911a018f4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f2278735-e384-4164-b664-cdd911a018f4" (UID: "f2278735-e384-4164-b664-cdd911a018f4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:53:10.945150 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:10.945019    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-dshm" (OuterVolumeSpecName: "dshm") pod "f2278735-e384-4164-b664-cdd911a018f4" (UID: "f2278735-e384-4164-b664-cdd911a018f4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:53:11.043662 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.043591    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l877\" (UniqueName: \"kubernetes.io/projected/f2278735-e384-4164-b664-cdd911a018f4-kube-api-access-8l877\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:53:11.043662 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.043654    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-home\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:53:11.043662 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.043670    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-tmp-dir\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:53:11.043925 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.043679    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-model-cache\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:53:11.043925 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.043688    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2278735-e384-4164-b664-cdd911a018f4-dshm\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:53:11.043925 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.043695    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2278735-e384-4164-b664-cdd911a018f4-tls-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:53:11.347450 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.347362    2581 generic.go:356] "Generic (PLEG): container finished" podID="f2278735-e384-4164-b664-cdd911a018f4" containerID="5170a1970f7e68e22bf78137cbe7f36d6e675a06860271f81f3a5d07d913f77a" exitCode=0
Jul 06 18:53:11.347450 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.347407    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"
Jul 06 18:53:11.347661 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.347443    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn" event={"ID":"f2278735-e384-4164-b664-cdd911a018f4","Type":"ContainerDied","Data":"5170a1970f7e68e22bf78137cbe7f36d6e675a06860271f81f3a5d07d913f77a"}
Jul 06 18:53:11.347661 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.347484    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn" event={"ID":"f2278735-e384-4164-b664-cdd911a018f4","Type":"ContainerDied","Data":"96d3fb83b441526d9ee13434f0317190d9b6c196ef940993607a3e58b23d59d0"}
Jul 06 18:53:11.347661 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.347501    2581 scope.go:117] "RemoveContainer" containerID="5170a1970f7e68e22bf78137cbe7f36d6e675a06860271f81f3a5d07d913f77a"
Jul 06 18:53:11.356359 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.356342    2581 scope.go:117] "RemoveContainer" containerID="5170a1970f7e68e22bf78137cbe7f36d6e675a06860271f81f3a5d07d913f77a"
Jul 06 18:53:11.356607 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:53:11.356590    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5170a1970f7e68e22bf78137cbe7f36d6e675a06860271f81f3a5d07d913f77a\": container with ID starting with 5170a1970f7e68e22bf78137cbe7f36d6e675a06860271f81f3a5d07d913f77a not found: ID does not exist" containerID="5170a1970f7e68e22bf78137cbe7f36d6e675a06860271f81f3a5d07d913f77a"
Jul 06 18:53:11.356649 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.356616    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5170a1970f7e68e22bf78137cbe7f36d6e675a06860271f81f3a5d07d913f77a"} err="failed to get container status \"5170a1970f7e68e22bf78137cbe7f36d6e675a06860271f81f3a5d07d913f77a\": rpc error: code = NotFound desc = could not find container \"5170a1970f7e68e22bf78137cbe7f36d6e675a06860271f81f3a5d07d913f77a\": container with ID starting with 5170a1970f7e68e22bf78137cbe7f36d6e675a06860271f81f3a5d07d913f77a not found: ID does not exist"
Jul 06 18:53:11.375007 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.374981    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"]
Jul 06 18:53:11.378306 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.378282    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-f2348c/test-model-f2348c-kserve-6b586fcbf-6rcjn"]
Jul 06 18:53:11.472113 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.472079    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2278735-e384-4164-b664-cdd911a018f4" path="/var/lib/kubelet/pods/f2278735-e384-4164-b664-cdd911a018f4/volumes"
Jul 06 18:53:11.611782 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.611658    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z" podUID="6207f293-aceb-425c-bff0-84abc5d646e5" containerName="main" containerID="cri-o://01471944513d11ceb054672d578e2af505c61501e967362eccfbde21fc5edf81" gracePeriod=45
Jul 06 18:53:11.861331 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.861306    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:53:11.952850 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.952805    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-dshm\") pod \"6207f293-aceb-425c-bff0-84abc5d646e5\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") "
Jul 06 18:53:11.953049 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.952886    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-home\") pod \"6207f293-aceb-425c-bff0-84abc5d646e5\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") "
Jul 06 18:53:11.953049 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.952918    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6207f293-aceb-425c-bff0-84abc5d646e5-tls-certs\") pod \"6207f293-aceb-425c-bff0-84abc5d646e5\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") "
Jul 06 18:53:11.953049 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.952950    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqxfg\" (UniqueName: \"kubernetes.io/projected/6207f293-aceb-425c-bff0-84abc5d646e5-kube-api-access-bqxfg\") pod \"6207f293-aceb-425c-bff0-84abc5d646e5\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") "
Jul 06 18:53:11.953049 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.952982    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-tmp-dir\") pod \"6207f293-aceb-425c-bff0-84abc5d646e5\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") "
Jul 06 18:53:11.953049 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.953002    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-model-cache\") pod \"6207f293-aceb-425c-bff0-84abc5d646e5\" (UID: \"6207f293-aceb-425c-bff0-84abc5d646e5\") "
Jul 06 18:53:11.953303 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.953159    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-home" (OuterVolumeSpecName: "home") pod "6207f293-aceb-425c-bff0-84abc5d646e5" (UID: "6207f293-aceb-425c-bff0-84abc5d646e5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:53:11.953303 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.953252    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-home\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:53:11.953383 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.953298    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6207f293-aceb-425c-bff0-84abc5d646e5" (UID: "6207f293-aceb-425c-bff0-84abc5d646e5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:53:11.953383 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.953343    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-model-cache" (OuterVolumeSpecName: "model-cache") pod "6207f293-aceb-425c-bff0-84abc5d646e5" (UID: "6207f293-aceb-425c-bff0-84abc5d646e5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:53:11.955040 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.955009    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6207f293-aceb-425c-bff0-84abc5d646e5-kube-api-access-bqxfg" (OuterVolumeSpecName: "kube-api-access-bqxfg") pod "6207f293-aceb-425c-bff0-84abc5d646e5" (UID: "6207f293-aceb-425c-bff0-84abc5d646e5"). InnerVolumeSpecName "kube-api-access-bqxfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:53:11.955040 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.955011    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6207f293-aceb-425c-bff0-84abc5d646e5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6207f293-aceb-425c-bff0-84abc5d646e5" (UID: "6207f293-aceb-425c-bff0-84abc5d646e5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:53:11.955188 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:11.955047    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-dshm" (OuterVolumeSpecName: "dshm") pod "6207f293-aceb-425c-bff0-84abc5d646e5" (UID: "6207f293-aceb-425c-bff0-84abc5d646e5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:53:12.054137 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:12.054095    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-dshm\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:53:12.054137 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:12.054128    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6207f293-aceb-425c-bff0-84abc5d646e5-tls-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:53:12.054137 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:12.054141    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqxfg\" (UniqueName: \"kubernetes.io/projected/6207f293-aceb-425c-bff0-84abc5d646e5-kube-api-access-bqxfg\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:53:12.054137 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:12.054148    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-tmp-dir\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:53:12.054403 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:12.054158    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6207f293-aceb-425c-bff0-84abc5d646e5-model-cache\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:53:12.359638 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:12.359470    2581 generic.go:356] "Generic (PLEG): container finished" podID="6207f293-aceb-425c-bff0-84abc5d646e5" containerID="01471944513d11ceb054672d578e2af505c61501e967362eccfbde21fc5edf81" exitCode=0
Jul 06 18:53:12.359638 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:12.359529    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z" event={"ID":"6207f293-aceb-425c-bff0-84abc5d646e5","Type":"ContainerDied","Data":"01471944513d11ceb054672d578e2af505c61501e967362eccfbde21fc5edf81"}
Jul 06 18:53:12.359865 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:12.359663    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z" event={"ID":"6207f293-aceb-425c-bff0-84abc5d646e5","Type":"ContainerDied","Data":"63ebadc7364c6b866638902f944dac46579c5a547384390a43bedcfa3e92f0ea"}
Jul 06 18:53:12.359865 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:12.359632    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"
Jul 06 18:53:12.359865 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:12.359684    2581 scope.go:117] "RemoveContainer" containerID="01471944513d11ceb054672d578e2af505c61501e967362eccfbde21fc5edf81"
Jul 06 18:53:12.369347 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:12.369329    2581 scope.go:117] "RemoveContainer" containerID="01471944513d11ceb054672d578e2af505c61501e967362eccfbde21fc5edf81"
Jul 06 18:53:12.369597 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:53:12.369577    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01471944513d11ceb054672d578e2af505c61501e967362eccfbde21fc5edf81\": container with ID starting with 01471944513d11ceb054672d578e2af505c61501e967362eccfbde21fc5edf81 not found: ID does not exist" containerID="01471944513d11ceb054672d578e2af505c61501e967362eccfbde21fc5edf81"
Jul 06 18:53:12.369635 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:12.369606    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01471944513d11ceb054672d578e2af505c61501e967362eccfbde21fc5edf81"} err="failed to get container status \"01471944513d11ceb054672d578e2af505c61501e967362eccfbde21fc5edf81\": rpc error: code = NotFound desc = could not find container \"01471944513d11ceb054672d578e2af505c61501e967362eccfbde21fc5edf81\": container with ID starting with 01471944513d11ceb054672d578e2af505c61501e967362eccfbde21fc5edf81 not found: ID does not exist"
Jul 06 18:53:12.390068 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:12.390038    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"]
Jul 06 18:53:12.391361 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:12.391342    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-ef5ef7/test-model-ef5ef7-kserve-6c8f5ddf4f-7bl9z"]
Jul 06 18:53:13.471881 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:13.471840    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6207f293-aceb-425c-bff0-84abc5d646e5" path="/var/lib/kubelet/pods/6207f293-aceb-425c-bff0-84abc5d646e5/volumes"
Jul 06 18:53:44.449722 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:44.449686    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:53:59.841324 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:53:59.841228    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:54:11.049605 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:11.049566    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:54:24.440787 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:24.440727    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:54:27.535067 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:27.535035    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:54:27.539220 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:27.539194    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:54:36.353670 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:36.353621    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:54:48.369956 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:48.369924    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:54:58.594665 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.594621    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"]
Jul 06 18:54:58.597026 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.595109    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6207f293-aceb-425c-bff0-84abc5d646e5" containerName="main"
Jul 06 18:54:58.597026 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.595127    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6207f293-aceb-425c-bff0-84abc5d646e5" containerName="main"
Jul 06 18:54:58.597026 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.595147    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2278735-e384-4164-b664-cdd911a018f4" containerName="main"
Jul 06 18:54:58.597026 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.595153    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2278735-e384-4164-b664-cdd911a018f4" containerName="main"
Jul 06 18:54:58.597026 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.595210    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6207f293-aceb-425c-bff0-84abc5d646e5" containerName="main"
Jul 06 18:54:58.597026 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.595219    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2278735-e384-4164-b664-cdd911a018f4" containerName="main"
Jul 06 18:54:58.597902 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.597884    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.606667 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.606644    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-47f1eb\"/\"kube-root-ca.crt\""
Jul 06 18:54:58.606771 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.606665    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-47f1eb\"/\"auth-test-model-47f1eb-kserve-self-signed-certs\""
Jul 06 18:54:58.606771 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.606646    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-47f1eb\"/\"openshift-service-ca.crt\""
Jul 06 18:54:58.606771 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.606646    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-47f1eb\"/\"default-dockercfg-wsdmw\""
Jul 06 18:54:58.619072 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.619048    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"]
Jul 06 18:54:58.660703 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.660672    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82063b8e-6697-47fb-a8a3-141a5c284085-tls-certs\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.660869 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.660710    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-dshm\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.660869 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.660753    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-home\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.660869 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.660800    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn6tm\" (UniqueName: \"kubernetes.io/projected/82063b8e-6697-47fb-a8a3-141a5c284085-kube-api-access-vn6tm\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.660869 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.660830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-model-cache\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.661038 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.660880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-tmp-dir\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.761582 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.761544    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-dshm\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.761582 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.761583    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-home\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.761855 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.761623    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn6tm\" (UniqueName: \"kubernetes.io/projected/82063b8e-6697-47fb-a8a3-141a5c284085-kube-api-access-vn6tm\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.761855 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.761682    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-model-cache\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.761855 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.761724    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-tmp-dir\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.761855 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.761787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82063b8e-6697-47fb-a8a3-141a5c284085-tls-certs\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.762133 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.762107    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-home\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.762199 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.762180    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-model-cache\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.762199 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.762190    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-tmp-dir\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.764036 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.764015    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-dshm\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.764314 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.764294    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82063b8e-6697-47fb-a8a3-141a5c284085-tls-certs\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.773550 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.773523    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn6tm\" (UniqueName: \"kubernetes.io/projected/82063b8e-6697-47fb-a8a3-141a5c284085-kube-api-access-vn6tm\") pod \"auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") " pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:58.906841 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:58.906805    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:54:59.058111 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:59.058073    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"]
Jul 06 18:54:59.061557 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:54:59.061523    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82063b8e_6697_47fb_a8a3_141a5c284085.slice/crio-dc69381c3d2086dfda4c16ce3bbc292032800458580385e80c6134994ee17b8d WatchSource:0}: Error finding container dc69381c3d2086dfda4c16ce3bbc292032800458580385e80c6134994ee17b8d: Status 404 returned error can't find the container with id dc69381c3d2086dfda4c16ce3bbc292032800458580385e80c6134994ee17b8d
Jul 06 18:54:59.841412 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:59.841376    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z" event={"ID":"82063b8e-6697-47fb-a8a3-141a5c284085","Type":"ContainerStarted","Data":"7f64035f65907126bfe0f8f0d836068079fc2d346c470895c504e3ba7fe810b6"}
Jul 06 18:54:59.841412 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:54:59.841424    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z" event={"ID":"82063b8e-6697-47fb-a8a3-141a5c284085","Type":"ContainerStarted","Data":"dc69381c3d2086dfda4c16ce3bbc292032800458580385e80c6134994ee17b8d"}
Jul 06 18:55:02.259409 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:02.259350    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z" podStartSLOduration=4.259332844 podStartE2EDuration="4.259332844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:54:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:54:59.872433122 +0000 UTC m=+2133.845882635" watchObservedRunningTime="2026-07-06 18:55:02.259332844 +0000 UTC m=+2136.232782357"
Jul 06 18:55:02.260747 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:02.260712    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:55:08.907899 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:08.907853    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:55:08.908584 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:08.907918    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:55:08.920517 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:08.920488    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:55:09.900442 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:09.900410    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:55:28.185900 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.185810    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"]
Jul 06 18:55:28.189341 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.189321    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.201003 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.200976    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b19850\"/\"kube-root-ca.crt\""
Jul 06 18:55:28.201165 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.201034    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b19850\"/\"default-dockercfg-c67nr\""
Jul 06 18:55:28.201165 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.201054    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b19850\"/\"openshift-service-ca.crt\""
Jul 06 18:55:28.201302 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.201288    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b19850\"/\"auth-test-model-b19850-kserve-self-signed-certs\""
Jul 06 18:55:28.211943 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.211915    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"]
Jul 06 18:55:28.322196 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.322155    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-home\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.322385 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.322202    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvwdz\" (UniqueName: \"kubernetes.io/projected/706be0f1-9911-47ea-99c3-c4217c4eb6c5-kube-api-access-qvwdz\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.322385 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.322317    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/706be0f1-9911-47ea-99c3-c4217c4eb6c5-tls-certs\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.322385 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.322351    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-dshm\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.322534 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.322391    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-model-cache\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.322534 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.322468    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-tmp-dir\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.423933 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.423897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-home\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.423933 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.423935    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvwdz\" (UniqueName: \"kubernetes.io/projected/706be0f1-9911-47ea-99c3-c4217c4eb6c5-kube-api-access-qvwdz\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.424188 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.423973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/706be0f1-9911-47ea-99c3-c4217c4eb6c5-tls-certs\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.424188 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.423988    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-dshm\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.424188 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.424013    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-model-cache\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.424188 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.424031    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-tmp-dir\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.424398 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.424368    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-home\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.424445 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.424399    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-tmp-dir\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.424508 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.424487    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-model-cache\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.426211 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.426186    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-dshm\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.426492 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.426472    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/706be0f1-9911-47ea-99c3-c4217c4eb6c5-tls-certs\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.435172 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.435138    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvwdz\" (UniqueName: \"kubernetes.io/projected/706be0f1-9911-47ea-99c3-c4217c4eb6c5-kube-api-access-qvwdz\") pod \"auth-test-model-b19850-kserve-84c76f5d47-g5mcd\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") " pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.499273 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.499189    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:28.639365 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.639298    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"]
Jul 06 18:55:28.641581 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:55:28.641543    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod706be0f1_9911_47ea_99c3_c4217c4eb6c5.slice/crio-e715656ab73395a4243c316f0953775673d00998e188f9d894856260873239ab WatchSource:0}: Error finding container e715656ab73395a4243c316f0953775673d00998e188f9d894856260873239ab: Status 404 returned error can't find the container with id e715656ab73395a4243c316f0953775673d00998e188f9d894856260873239ab
Jul 06 18:55:28.973073 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.973027    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd" event={"ID":"706be0f1-9911-47ea-99c3-c4217c4eb6c5","Type":"ContainerStarted","Data":"79485866be4309aa40300660d6d9eb04059c4f9b284ed38060eb31caacc6a3ca"}
Jul 06 18:55:28.973073 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.973079    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd" event={"ID":"706be0f1-9911-47ea-99c3-c4217c4eb6c5","Type":"ContainerStarted","Data":"e715656ab73395a4243c316f0953775673d00998e188f9d894856260873239ab"}
Jul 06 18:55:28.998123 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:28.998072    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd" podStartSLOduration=0.998054002 podStartE2EDuration="998.054002ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:55:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:55:28.997127384 +0000 UTC m=+2162.970576893" watchObservedRunningTime="2026-07-06 18:55:28.998054002 +0000 UTC m=+2162.971503515"
Jul 06 18:55:32.140806 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:32.140770    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:55:38.499815 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:38.499771    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:38.500338 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:38.499857    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:38.512942 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:38.512916    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:39.030863 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:39.030834    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:55:46.547018 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:55:46.546983    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:56:00.457503 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:56:00.457466    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:56:19.559761 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:56:19.556639    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:56:35.642206 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:56:35.642164    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:56:52.743190 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:56:52.743103    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:57:08.970460 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:08.970420    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:57:25.960484 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:25.960441    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:57:36.151826 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:36.151789    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"]
Jul 06 18:57:37.125395 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:37.125350    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"]
Jul 06 18:57:40.941174 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:40.941136    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:57:42.049340 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.049306    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"]
Jul 06 18:57:42.052665 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.052647    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.055758 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.055718    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-47f1eb\"/\"sub-test-model-47f1eb-kserve-self-signed-certs\""
Jul 06 18:57:42.062395 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.062370    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"]
Jul 06 18:57:42.196410 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.196368    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-home\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.196596 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.196430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-model-cache\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.196596 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.196518    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-tmp-dir\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.196596 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.196551    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-dshm\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.196596 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.196591    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kdkv\" (UniqueName: \"kubernetes.io/projected/1195cd91-3196-4506-9a4d-cfe3f673c1e6-kube-api-access-2kdkv\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.196797 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.196664    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1195cd91-3196-4506-9a4d-cfe3f673c1e6-tls-certs\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.298208 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.298164    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-tmp-dir\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.298208 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.298214    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-dshm\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.298452 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.298239    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kdkv\" (UniqueName: \"kubernetes.io/projected/1195cd91-3196-4506-9a4d-cfe3f673c1e6-kube-api-access-2kdkv\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.298452 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.298327    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1195cd91-3196-4506-9a4d-cfe3f673c1e6-tls-certs\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.298452 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.298406    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-home\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.298610 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.298463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-model-cache\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.298667 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.298629    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-tmp-dir\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.298827 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.298805    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-home\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.298996 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.298976    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-model-cache\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.300635 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.300582    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-dshm\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.300948 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.300929    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1195cd91-3196-4506-9a4d-cfe3f673c1e6-tls-certs\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.316907 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.316880    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kdkv\" (UniqueName: \"kubernetes.io/projected/1195cd91-3196-4506-9a4d-cfe3f673c1e6-kube-api-access-2kdkv\") pod \"sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") " pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.362693 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.362656    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:42.506506 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.506480    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"]
Jul 06 18:57:42.509130 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:57:42.509095    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1195cd91_3196_4506_9a4d_cfe3f673c1e6.slice/crio-0299e2719834b3ffeb7aa4824c3af56299469524beea151eda44b4988e47a9b3 WatchSource:0}: Error finding container 0299e2719834b3ffeb7aa4824c3af56299469524beea151eda44b4988e47a9b3: Status 404 returned error can't find the container with id 0299e2719834b3ffeb7aa4824c3af56299469524beea151eda44b4988e47a9b3
Jul 06 18:57:42.604765 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.604699    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr" event={"ID":"1195cd91-3196-4506-9a4d-cfe3f673c1e6","Type":"ContainerStarted","Data":"5574cb8d1ac018578dec0a67093ba7d561c03a8eba3190186c33451f8e22ebb4"}
Jul 06 18:57:42.604938 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.604774    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr" event={"ID":"1195cd91-3196-4506-9a4d-cfe3f673c1e6","Type":"ContainerStarted","Data":"0299e2719834b3ffeb7aa4824c3af56299469524beea151eda44b4988e47a9b3"}
Jul 06 18:57:42.623249 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:42.623182    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr" podStartSLOduration=0.623161107 podStartE2EDuration="623.161107ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:57:42.621643536 +0000 UTC m=+2296.595093052" watchObservedRunningTime="2026-07-06 18:57:42.623161107 +0000 UTC m=+2296.596610621"
Jul 06 18:57:51.172682 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.172616    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z" podUID="82063b8e-6697-47fb-a8a3-141a5c284085" containerName="main" containerID="cri-o://7f64035f65907126bfe0f8f0d836068079fc2d346c470895c504e3ba7fe810b6" gracePeriod=45
Jul 06 18:57:51.422359 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.422329    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:57:51.578460 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.578356    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-dshm\") pod \"82063b8e-6697-47fb-a8a3-141a5c284085\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") "
Jul 06 18:57:51.578460 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.578429    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vn6tm\" (UniqueName: \"kubernetes.io/projected/82063b8e-6697-47fb-a8a3-141a5c284085-kube-api-access-vn6tm\") pod \"82063b8e-6697-47fb-a8a3-141a5c284085\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") "
Jul 06 18:57:51.578460 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.578456    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82063b8e-6697-47fb-a8a3-141a5c284085-tls-certs\") pod \"82063b8e-6697-47fb-a8a3-141a5c284085\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") "
Jul 06 18:57:51.578786 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.578514    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-home\") pod \"82063b8e-6697-47fb-a8a3-141a5c284085\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") "
Jul 06 18:57:51.578786 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.578533    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-tmp-dir\") pod \"82063b8e-6697-47fb-a8a3-141a5c284085\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") "
Jul 06 18:57:51.578786 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.578774    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-model-cache\") pod \"82063b8e-6697-47fb-a8a3-141a5c284085\" (UID: \"82063b8e-6697-47fb-a8a3-141a5c284085\") "
Jul 06 18:57:51.578968 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.578821    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "82063b8e-6697-47fb-a8a3-141a5c284085" (UID: "82063b8e-6697-47fb-a8a3-141a5c284085"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:57:51.578968 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.578852    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-home" (OuterVolumeSpecName: "home") pod "82063b8e-6697-47fb-a8a3-141a5c284085" (UID: "82063b8e-6697-47fb-a8a3-141a5c284085"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:57:51.579049 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.579015    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-model-cache" (OuterVolumeSpecName: "model-cache") pod "82063b8e-6697-47fb-a8a3-141a5c284085" (UID: "82063b8e-6697-47fb-a8a3-141a5c284085"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:57:51.579134 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.579108    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-home\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:57:51.579134 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.579129    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-tmp-dir\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:57:51.579210 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.579144    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-model-cache\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:57:51.580753 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.580717    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82063b8e-6697-47fb-a8a3-141a5c284085-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "82063b8e-6697-47fb-a8a3-141a5c284085" (UID: "82063b8e-6697-47fb-a8a3-141a5c284085"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:57:51.580844 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.580793    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-dshm" (OuterVolumeSpecName: "dshm") pod "82063b8e-6697-47fb-a8a3-141a5c284085" (UID: "82063b8e-6697-47fb-a8a3-141a5c284085"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:57:51.580903 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.580856    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82063b8e-6697-47fb-a8a3-141a5c284085-kube-api-access-vn6tm" (OuterVolumeSpecName: "kube-api-access-vn6tm") pod "82063b8e-6697-47fb-a8a3-141a5c284085" (UID: "82063b8e-6697-47fb-a8a3-141a5c284085"). InnerVolumeSpecName "kube-api-access-vn6tm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:57:51.649917 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.649879    2581 generic.go:356] "Generic (PLEG): container finished" podID="82063b8e-6697-47fb-a8a3-141a5c284085" containerID="7f64035f65907126bfe0f8f0d836068079fc2d346c470895c504e3ba7fe810b6" exitCode=0
Jul 06 18:57:51.650066 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.649937    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"
Jul 06 18:57:51.650066 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.649972    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z" event={"ID":"82063b8e-6697-47fb-a8a3-141a5c284085","Type":"ContainerDied","Data":"7f64035f65907126bfe0f8f0d836068079fc2d346c470895c504e3ba7fe810b6"}
Jul 06 18:57:51.650066 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.650017    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z" event={"ID":"82063b8e-6697-47fb-a8a3-141a5c284085","Type":"ContainerDied","Data":"dc69381c3d2086dfda4c16ce3bbc292032800458580385e80c6134994ee17b8d"}
Jul 06 18:57:51.650066 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.650035    2581 scope.go:117] "RemoveContainer" containerID="7f64035f65907126bfe0f8f0d836068079fc2d346c470895c504e3ba7fe810b6"
Jul 06 18:57:51.659053 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.659037    2581 scope.go:117] "RemoveContainer" containerID="7f64035f65907126bfe0f8f0d836068079fc2d346c470895c504e3ba7fe810b6"
Jul 06 18:57:51.659283 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:57:51.659267    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f64035f65907126bfe0f8f0d836068079fc2d346c470895c504e3ba7fe810b6\": container with ID starting with 7f64035f65907126bfe0f8f0d836068079fc2d346c470895c504e3ba7fe810b6 not found: ID does not exist" containerID="7f64035f65907126bfe0f8f0d836068079fc2d346c470895c504e3ba7fe810b6"
Jul 06 18:57:51.659340 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.659291    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f64035f65907126bfe0f8f0d836068079fc2d346c470895c504e3ba7fe810b6"} err="failed to get container status \"7f64035f65907126bfe0f8f0d836068079fc2d346c470895c504e3ba7fe810b6\": rpc error: code = NotFound desc = could not find container \"7f64035f65907126bfe0f8f0d836068079fc2d346c470895c504e3ba7fe810b6\": container with ID starting with 7f64035f65907126bfe0f8f0d836068079fc2d346c470895c504e3ba7fe810b6 not found: ID does not exist"
Jul 06 18:57:51.675632 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.675608    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"]
Jul 06 18:57:51.680102 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.680068    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82063b8e-6697-47fb-a8a3-141a5c284085-dshm\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:57:51.680102 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.680091    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vn6tm\" (UniqueName: \"kubernetes.io/projected/82063b8e-6697-47fb-a8a3-141a5c284085-kube-api-access-vn6tm\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:57:51.680102 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.680102    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82063b8e-6697-47fb-a8a3-141a5c284085-tls-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:57:51.682709 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:51.682685    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/auth-test-model-47f1eb-kserve-6b74ff8fc5-ccs2z"]
Jul 06 18:57:52.145240 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.145176    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd" podUID="706be0f1-9911-47ea-99c3-c4217c4eb6c5" containerName="main" containerID="cri-o://79485866be4309aa40300660d6d9eb04059c4f9b284ed38060eb31caacc6a3ca" gracePeriod=45
Jul 06 18:57:52.363140 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.363102    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:52.363522 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.363192    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:52.375920 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.375893    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:52.404329 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.404266    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:57:52.588878 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.588845    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-model-cache\") pod \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") "
Jul 06 18:57:52.589087 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.588907    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-home\") pod \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") "
Jul 06 18:57:52.589087 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.588950    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvwdz\" (UniqueName: \"kubernetes.io/projected/706be0f1-9911-47ea-99c3-c4217c4eb6c5-kube-api-access-qvwdz\") pod \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") "
Jul 06 18:57:52.589087 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.588973    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/706be0f1-9911-47ea-99c3-c4217c4eb6c5-tls-certs\") pod \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") "
Jul 06 18:57:52.589087 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.589019    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-tmp-dir\") pod \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") "
Jul 06 18:57:52.589087 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.589069    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-dshm\") pod \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\" (UID: \"706be0f1-9911-47ea-99c3-c4217c4eb6c5\") "
Jul 06 18:57:52.589305 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.589183    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-model-cache" (OuterVolumeSpecName: "model-cache") pod "706be0f1-9911-47ea-99c3-c4217c4eb6c5" (UID: "706be0f1-9911-47ea-99c3-c4217c4eb6c5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:57:52.589305 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.589205    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-home" (OuterVolumeSpecName: "home") pod "706be0f1-9911-47ea-99c3-c4217c4eb6c5" (UID: "706be0f1-9911-47ea-99c3-c4217c4eb6c5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:57:52.589376 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.589344    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "706be0f1-9911-47ea-99c3-c4217c4eb6c5" (UID: "706be0f1-9911-47ea-99c3-c4217c4eb6c5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:57:52.589416 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.589375    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-model-cache\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:57:52.589416 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.589391    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-home\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:57:52.591900 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.591870    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/706be0f1-9911-47ea-99c3-c4217c4eb6c5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "706be0f1-9911-47ea-99c3-c4217c4eb6c5" (UID: "706be0f1-9911-47ea-99c3-c4217c4eb6c5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:57:52.592001 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.591902    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/706be0f1-9911-47ea-99c3-c4217c4eb6c5-kube-api-access-qvwdz" (OuterVolumeSpecName: "kube-api-access-qvwdz") pod "706be0f1-9911-47ea-99c3-c4217c4eb6c5" (UID: "706be0f1-9911-47ea-99c3-c4217c4eb6c5"). InnerVolumeSpecName "kube-api-access-qvwdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:57:52.592001 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.591992    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-dshm" (OuterVolumeSpecName: "dshm") pod "706be0f1-9911-47ea-99c3-c4217c4eb6c5" (UID: "706be0f1-9911-47ea-99c3-c4217c4eb6c5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:57:52.656870 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.656779    2581 generic.go:356] "Generic (PLEG): container finished" podID="706be0f1-9911-47ea-99c3-c4217c4eb6c5" containerID="79485866be4309aa40300660d6d9eb04059c4f9b284ed38060eb31caacc6a3ca" exitCode=0
Jul 06 18:57:52.656870 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.656824    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"
Jul 06 18:57:52.657072 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.656870    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd" event={"ID":"706be0f1-9911-47ea-99c3-c4217c4eb6c5","Type":"ContainerDied","Data":"79485866be4309aa40300660d6d9eb04059c4f9b284ed38060eb31caacc6a3ca"}
Jul 06 18:57:52.657072 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.656912    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd" event={"ID":"706be0f1-9911-47ea-99c3-c4217c4eb6c5","Type":"ContainerDied","Data":"e715656ab73395a4243c316f0953775673d00998e188f9d894856260873239ab"}
Jul 06 18:57:52.657072 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.656933    2581 scope.go:117] "RemoveContainer" containerID="79485866be4309aa40300660d6d9eb04059c4f9b284ed38060eb31caacc6a3ca"
Jul 06 18:57:52.668111 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.668088    2581 scope.go:117] "RemoveContainer" containerID="79485866be4309aa40300660d6d9eb04059c4f9b284ed38060eb31caacc6a3ca"
Jul 06 18:57:52.668418 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:57:52.668396    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79485866be4309aa40300660d6d9eb04059c4f9b284ed38060eb31caacc6a3ca\": container with ID starting with 79485866be4309aa40300660d6d9eb04059c4f9b284ed38060eb31caacc6a3ca not found: ID does not exist" containerID="79485866be4309aa40300660d6d9eb04059c4f9b284ed38060eb31caacc6a3ca"
Jul 06 18:57:52.668531 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.668429    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79485866be4309aa40300660d6d9eb04059c4f9b284ed38060eb31caacc6a3ca"} err="failed to get container status \"79485866be4309aa40300660d6d9eb04059c4f9b284ed38060eb31caacc6a3ca\": rpc error: code = NotFound desc = could not find container \"79485866be4309aa40300660d6d9eb04059c4f9b284ed38060eb31caacc6a3ca\": container with ID starting with 79485866be4309aa40300660d6d9eb04059c4f9b284ed38060eb31caacc6a3ca not found: ID does not exist"
Jul 06 18:57:52.672952 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.672930    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:57:52.683580 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.683526    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"]
Jul 06 18:57:52.685562 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.685534    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b19850/auth-test-model-b19850-kserve-84c76f5d47-g5mcd"]
Jul 06 18:57:52.690604 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.690584    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvwdz\" (UniqueName: \"kubernetes.io/projected/706be0f1-9911-47ea-99c3-c4217c4eb6c5-kube-api-access-qvwdz\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:57:52.690715 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.690607    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/706be0f1-9911-47ea-99c3-c4217c4eb6c5-tls-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:57:52.690715 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.690621    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-tmp-dir\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:57:52.690715 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:52.690633    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/706be0f1-9911-47ea-99c3-c4217c4eb6c5-dshm\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:57:53.471791 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:53.471756    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="706be0f1-9911-47ea-99c3-c4217c4eb6c5" path="/var/lib/kubelet/pods/706be0f1-9911-47ea-99c3-c4217c4eb6c5/volumes"
Jul 06 18:57:53.472151 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:53.472138    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82063b8e-6697-47fb-a8a3-141a5c284085" path="/var/lib/kubelet/pods/82063b8e-6697-47fb-a8a3-141a5c284085/volumes"
Jul 06 18:57:57.440061 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:57:57.440020    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:58:12.787035 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.787004    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"]
Jul 06 18:58:12.787530 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.787331    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82063b8e-6697-47fb-a8a3-141a5c284085" containerName="main"
Jul 06 18:58:12.787530 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.787345    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="82063b8e-6697-47fb-a8a3-141a5c284085" containerName="main"
Jul 06 18:58:12.787530 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.787368    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="706be0f1-9911-47ea-99c3-c4217c4eb6c5" containerName="main"
Jul 06 18:58:12.787530 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.787373    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="706be0f1-9911-47ea-99c3-c4217c4eb6c5" containerName="main"
Jul 06 18:58:12.787530 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.787440    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="82063b8e-6697-47fb-a8a3-141a5c284085" containerName="main"
Jul 06 18:58:12.787530 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.787451    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="706be0f1-9911-47ea-99c3-c4217c4eb6c5" containerName="main"
Jul 06 18:58:12.790255 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.790236    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.796842 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.796822    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b19850\"/\"default-dockercfg-c67nr\""
Jul 06 18:58:12.797754 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.797715    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b19850\"/\"kube-root-ca.crt\""
Jul 06 18:58:12.797860 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.797781    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b19850\"/\"openshift-service-ca.crt\""
Jul 06 18:58:12.797860 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.797715    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b19850\"/\"sub-test-model-b19850-kserve-self-signed-certs\""
Jul 06 18:58:12.815130 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.815098    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"]
Jul 06 18:58:12.863035 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.863003    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtmmn\" (UniqueName: \"kubernetes.io/projected/383317eb-2a54-4351-abd3-91cd42c0fbe7-kube-api-access-rtmmn\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.863215 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.863042    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/383317eb-2a54-4351-abd3-91cd42c0fbe7-tls-certs\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.863215 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.863061    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-home\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.863215 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.863086    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-dshm\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.863215 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.863144    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-tmp-dir\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.863215 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.863164    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-model-cache\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.964204 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.964162    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-tmp-dir\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.964204 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.964206    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-model-cache\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.964460 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.964242    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtmmn\" (UniqueName: \"kubernetes.io/projected/383317eb-2a54-4351-abd3-91cd42c0fbe7-kube-api-access-rtmmn\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.964460 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.964265    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/383317eb-2a54-4351-abd3-91cd42c0fbe7-tls-certs\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.964460 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.964281    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-home\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.964460 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.964307    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-dshm\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.964695 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.964623    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-tmp-dir\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.964695 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.964671    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-model-cache\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.964695 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.964683    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-home\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.966694 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.966674    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-dshm\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.966888 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.966871    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/383317eb-2a54-4351-abd3-91cd42c0fbe7-tls-certs\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:12.974509 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:12.974486    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtmmn\" (UniqueName: \"kubernetes.io/projected/383317eb-2a54-4351-abd3-91cd42c0fbe7-kube-api-access-rtmmn\") pod \"sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") " pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:13.099127 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:13.099037    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:13.242015 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:13.241987    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"]
Jul 06 18:58:13.244563 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:58:13.244536    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod383317eb_2a54_4351_abd3_91cd42c0fbe7.slice/crio-2ad54e14e1aed0768f42ed55674db217db2500a5dedefc82e6c0ebcddae29710 WatchSource:0}: Error finding container 2ad54e14e1aed0768f42ed55674db217db2500a5dedefc82e6c0ebcddae29710: Status 404 returned error can't find the container with id 2ad54e14e1aed0768f42ed55674db217db2500a5dedefc82e6c0ebcddae29710
Jul 06 18:58:13.761423 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:13.761384    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm" event={"ID":"383317eb-2a54-4351-abd3-91cd42c0fbe7","Type":"ContainerStarted","Data":"e6d78c0dce021bd108fecb7fd54ee625e6b1aa5c5fe2778f5314e5974fb6b790"}
Jul 06 18:58:13.761423 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:13.761425    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm" event={"ID":"383317eb-2a54-4351-abd3-91cd42c0fbe7","Type":"ContainerStarted","Data":"2ad54e14e1aed0768f42ed55674db217db2500a5dedefc82e6c0ebcddae29710"}
Jul 06 18:58:13.780082 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:13.780031    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm" podStartSLOduration=1.7800169970000002 podStartE2EDuration="1.780016997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:58:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:58:13.778999223 +0000 UTC m=+2327.752448734" watchObservedRunningTime="2026-07-06 18:58:13.780016997 +0000 UTC m=+2327.753466511"
Jul 06 18:58:17.489781 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:17.489730    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:58:23.099967 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:23.099874    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:23.100515 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:23.099966    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:23.112563 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:23.112538    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:23.821841 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:23.821813    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:58:30.954816 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:30.954778    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:58:50.370705 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:50.370660    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:58:56.042607 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:56.042570    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"]
Jul 06 18:58:57.044722 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:58:57.044686    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"]
Jul 06 18:59:07.875270 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:07.875236    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:59:11.062498 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.062442    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr" podUID="1195cd91-3196-4506-9a4d-cfe3f673c1e6" containerName="main" containerID="cri-o://5574cb8d1ac018578dec0a67093ba7d561c03a8eba3190186c33451f8e22ebb4" gracePeriod=45
Jul 06 18:59:11.320998 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.320934    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:59:11.369065 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.369028    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-model-cache\") pod \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") "
Jul 06 18:59:11.369228 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.369083    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-dshm\") pod \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") "
Jul 06 18:59:11.369228 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.369106    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-tmp-dir\") pod \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") "
Jul 06 18:59:11.369228 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.369129    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1195cd91-3196-4506-9a4d-cfe3f673c1e6-tls-certs\") pod \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") "
Jul 06 18:59:11.369228 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.369158    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kdkv\" (UniqueName: \"kubernetes.io/projected/1195cd91-3196-4506-9a4d-cfe3f673c1e6-kube-api-access-2kdkv\") pod \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") "
Jul 06 18:59:11.369440 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.369233    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-home\") pod \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\" (UID: \"1195cd91-3196-4506-9a4d-cfe3f673c1e6\") "
Jul 06 18:59:11.369440 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.369384    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1195cd91-3196-4506-9a4d-cfe3f673c1e6" (UID: "1195cd91-3196-4506-9a4d-cfe3f673c1e6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:59:11.369440 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.369399    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-model-cache" (OuterVolumeSpecName: "model-cache") pod "1195cd91-3196-4506-9a4d-cfe3f673c1e6" (UID: "1195cd91-3196-4506-9a4d-cfe3f673c1e6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:59:11.369619 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.369600    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-home" (OuterVolumeSpecName: "home") pod "1195cd91-3196-4506-9a4d-cfe3f673c1e6" (UID: "1195cd91-3196-4506-9a4d-cfe3f673c1e6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:59:11.371299 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.371273    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1195cd91-3196-4506-9a4d-cfe3f673c1e6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1195cd91-3196-4506-9a4d-cfe3f673c1e6" (UID: "1195cd91-3196-4506-9a4d-cfe3f673c1e6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:59:11.371689 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.371666    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-dshm" (OuterVolumeSpecName: "dshm") pod "1195cd91-3196-4506-9a4d-cfe3f673c1e6" (UID: "1195cd91-3196-4506-9a4d-cfe3f673c1e6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:59:11.371772 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.371682    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1195cd91-3196-4506-9a4d-cfe3f673c1e6-kube-api-access-2kdkv" (OuterVolumeSpecName: "kube-api-access-2kdkv") pod "1195cd91-3196-4506-9a4d-cfe3f673c1e6" (UID: "1195cd91-3196-4506-9a4d-cfe3f673c1e6"). InnerVolumeSpecName "kube-api-access-2kdkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:59:11.470038 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.470009    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1195cd91-3196-4506-9a4d-cfe3f673c1e6-tls-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:59:11.470038 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.470035    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kdkv\" (UniqueName: \"kubernetes.io/projected/1195cd91-3196-4506-9a4d-cfe3f673c1e6-kube-api-access-2kdkv\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:59:11.470038 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.470045    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-home\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:59:11.470278 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.470055    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-model-cache\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:59:11.470278 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.470064    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-dshm\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:59:11.470278 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:11.470072    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1195cd91-3196-4506-9a4d-cfe3f673c1e6-tmp-dir\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:59:12.030534 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.030486    2581 generic.go:356] "Generic (PLEG): container finished" podID="1195cd91-3196-4506-9a4d-cfe3f673c1e6" containerID="5574cb8d1ac018578dec0a67093ba7d561c03a8eba3190186c33451f8e22ebb4" exitCode=0
Jul 06 18:59:12.030534 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.030533    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"
Jul 06 18:59:12.030794 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.030559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr" event={"ID":"1195cd91-3196-4506-9a4d-cfe3f673c1e6","Type":"ContainerDied","Data":"5574cb8d1ac018578dec0a67093ba7d561c03a8eba3190186c33451f8e22ebb4"}
Jul 06 18:59:12.030794 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.030604    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr" event={"ID":"1195cd91-3196-4506-9a4d-cfe3f673c1e6","Type":"ContainerDied","Data":"0299e2719834b3ffeb7aa4824c3af56299469524beea151eda44b4988e47a9b3"}
Jul 06 18:59:12.030794 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.030622    2581 scope.go:117] "RemoveContainer" containerID="5574cb8d1ac018578dec0a67093ba7d561c03a8eba3190186c33451f8e22ebb4"
Jul 06 18:59:12.039091 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.039063    2581 scope.go:117] "RemoveContainer" containerID="5574cb8d1ac018578dec0a67093ba7d561c03a8eba3190186c33451f8e22ebb4"
Jul 06 18:59:12.039372 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:59:12.039352    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5574cb8d1ac018578dec0a67093ba7d561c03a8eba3190186c33451f8e22ebb4\": container with ID starting with 5574cb8d1ac018578dec0a67093ba7d561c03a8eba3190186c33451f8e22ebb4 not found: ID does not exist" containerID="5574cb8d1ac018578dec0a67093ba7d561c03a8eba3190186c33451f8e22ebb4"
Jul 06 18:59:12.039431 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.039382    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5574cb8d1ac018578dec0a67093ba7d561c03a8eba3190186c33451f8e22ebb4"} err="failed to get container status \"5574cb8d1ac018578dec0a67093ba7d561c03a8eba3190186c33451f8e22ebb4\": rpc error: code = NotFound desc = could not find container \"5574cb8d1ac018578dec0a67093ba7d561c03a8eba3190186c33451f8e22ebb4\": container with ID starting with 5574cb8d1ac018578dec0a67093ba7d561c03a8eba3190186c33451f8e22ebb4 not found: ID does not exist"
Jul 06 18:59:12.055013 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.054983    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"]
Jul 06 18:59:12.060419 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.060394    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/sub-test-model-47f1eb-kserve-65cc59f5f7-q5rtr"]
Jul 06 18:59:12.064753 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.064713    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm" podUID="383317eb-2a54-4351-abd3-91cd42c0fbe7" containerName="main" containerID="cri-o://e6d78c0dce021bd108fecb7fd54ee625e6b1aa5c5fe2778f5314e5974fb6b790" gracePeriod=45
Jul 06 18:59:12.324380 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.324352    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:59:12.378638 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.378602    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-tmp-dir\") pod \"383317eb-2a54-4351-abd3-91cd42c0fbe7\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") "
Jul 06 18:59:12.378638 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.378644    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/383317eb-2a54-4351-abd3-91cd42c0fbe7-tls-certs\") pod \"383317eb-2a54-4351-abd3-91cd42c0fbe7\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") "
Jul 06 18:59:12.378926 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.378663    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-dshm\") pod \"383317eb-2a54-4351-abd3-91cd42c0fbe7\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") "
Jul 06 18:59:12.378926 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.378677    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-home\") pod \"383317eb-2a54-4351-abd3-91cd42c0fbe7\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") "
Jul 06 18:59:12.378926 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.378712    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtmmn\" (UniqueName: \"kubernetes.io/projected/383317eb-2a54-4351-abd3-91cd42c0fbe7-kube-api-access-rtmmn\") pod \"383317eb-2a54-4351-abd3-91cd42c0fbe7\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") "
Jul 06 18:59:12.378926 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.378854    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-model-cache\") pod \"383317eb-2a54-4351-abd3-91cd42c0fbe7\" (UID: \"383317eb-2a54-4351-abd3-91cd42c0fbe7\") "
Jul 06 18:59:12.379124 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.378980    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "383317eb-2a54-4351-abd3-91cd42c0fbe7" (UID: "383317eb-2a54-4351-abd3-91cd42c0fbe7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:59:12.379124 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.378991    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-home" (OuterVolumeSpecName: "home") pod "383317eb-2a54-4351-abd3-91cd42c0fbe7" (UID: "383317eb-2a54-4351-abd3-91cd42c0fbe7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:59:12.379124 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.379103    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-tmp-dir\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:59:12.379124 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.379120    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-home\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:59:12.379287 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.379136    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-model-cache" (OuterVolumeSpecName: "model-cache") pod "383317eb-2a54-4351-abd3-91cd42c0fbe7" (UID: "383317eb-2a54-4351-abd3-91cd42c0fbe7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:59:12.380858 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.380820    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-dshm" (OuterVolumeSpecName: "dshm") pod "383317eb-2a54-4351-abd3-91cd42c0fbe7" (UID: "383317eb-2a54-4351-abd3-91cd42c0fbe7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:59:12.380959 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.380866    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/383317eb-2a54-4351-abd3-91cd42c0fbe7-kube-api-access-rtmmn" (OuterVolumeSpecName: "kube-api-access-rtmmn") pod "383317eb-2a54-4351-abd3-91cd42c0fbe7" (UID: "383317eb-2a54-4351-abd3-91cd42c0fbe7"). InnerVolumeSpecName "kube-api-access-rtmmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:59:12.380959 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.380920    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/383317eb-2a54-4351-abd3-91cd42c0fbe7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "383317eb-2a54-4351-abd3-91cd42c0fbe7" (UID: "383317eb-2a54-4351-abd3-91cd42c0fbe7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:59:12.479845 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.479808    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-model-cache\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:59:12.479845 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.479840    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/383317eb-2a54-4351-abd3-91cd42c0fbe7-tls-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:59:12.479845 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.479848    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/383317eb-2a54-4351-abd3-91cd42c0fbe7-dshm\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:59:12.480075 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:12.479859    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtmmn\" (UniqueName: \"kubernetes.io/projected/383317eb-2a54-4351-abd3-91cd42c0fbe7-kube-api-access-rtmmn\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 18:59:13.037627 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:13.037598    2581 generic.go:356] "Generic (PLEG): container finished" podID="383317eb-2a54-4351-abd3-91cd42c0fbe7" containerID="e6d78c0dce021bd108fecb7fd54ee625e6b1aa5c5fe2778f5314e5974fb6b790" exitCode=0
Jul 06 18:59:13.037834 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:13.037637    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"
Jul 06 18:59:13.037834 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:13.037681    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm" event={"ID":"383317eb-2a54-4351-abd3-91cd42c0fbe7","Type":"ContainerDied","Data":"e6d78c0dce021bd108fecb7fd54ee625e6b1aa5c5fe2778f5314e5974fb6b790"}
Jul 06 18:59:13.037834 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:13.037725    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm" event={"ID":"383317eb-2a54-4351-abd3-91cd42c0fbe7","Type":"ContainerDied","Data":"2ad54e14e1aed0768f42ed55674db217db2500a5dedefc82e6c0ebcddae29710"}
Jul 06 18:59:13.037834 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:13.037764    2581 scope.go:117] "RemoveContainer" containerID="e6d78c0dce021bd108fecb7fd54ee625e6b1aa5c5fe2778f5314e5974fb6b790"
Jul 06 18:59:13.046606 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:13.046587    2581 scope.go:117] "RemoveContainer" containerID="e6d78c0dce021bd108fecb7fd54ee625e6b1aa5c5fe2778f5314e5974fb6b790"
Jul 06 18:59:13.046908 ip-10-0-141-229 kubenswrapper[2581]: E0706 18:59:13.046891    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6d78c0dce021bd108fecb7fd54ee625e6b1aa5c5fe2778f5314e5974fb6b790\": container with ID starting with e6d78c0dce021bd108fecb7fd54ee625e6b1aa5c5fe2778f5314e5974fb6b790 not found: ID does not exist" containerID="e6d78c0dce021bd108fecb7fd54ee625e6b1aa5c5fe2778f5314e5974fb6b790"
Jul 06 18:59:13.046962 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:13.046918    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6d78c0dce021bd108fecb7fd54ee625e6b1aa5c5fe2778f5314e5974fb6b790"} err="failed to get container status \"e6d78c0dce021bd108fecb7fd54ee625e6b1aa5c5fe2778f5314e5974fb6b790\": rpc error: code = NotFound desc = could not find container \"e6d78c0dce021bd108fecb7fd54ee625e6b1aa5c5fe2778f5314e5974fb6b790\": container with ID starting with e6d78c0dce021bd108fecb7fd54ee625e6b1aa5c5fe2778f5314e5974fb6b790 not found: ID does not exist"
Jul 06 18:59:13.074114 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:13.074076    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"]
Jul 06 18:59:13.077866 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:13.077837    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b19850/sub-test-model-b19850-kserve-8d9ff8bd9-wvxbm"]
Jul 06 18:59:13.471729 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:13.471690    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1195cd91-3196-4506-9a4d-cfe3f673c1e6" path="/var/lib/kubelet/pods/1195cd91-3196-4506-9a4d-cfe3f673c1e6/volumes"
Jul 06 18:59:13.472202 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:13.472180    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="383317eb-2a54-4351-abd3-91cd42c0fbe7" path="/var/lib/kubelet/pods/383317eb-2a54-4351-abd3-91cd42c0fbe7/volumes"
Jul 06 18:59:20.023615 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.023567    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"]
Jul 06 18:59:20.024135 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.023974    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1195cd91-3196-4506-9a4d-cfe3f673c1e6" containerName="main"
Jul 06 18:59:20.024135 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.023986    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1195cd91-3196-4506-9a4d-cfe3f673c1e6" containerName="main"
Jul 06 18:59:20.024135 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.023999    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="383317eb-2a54-4351-abd3-91cd42c0fbe7" containerName="main"
Jul 06 18:59:20.024135 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.024004    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="383317eb-2a54-4351-abd3-91cd42c0fbe7" containerName="main"
Jul 06 18:59:20.024135 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.024070    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="383317eb-2a54-4351-abd3-91cd42c0fbe7" containerName="main"
Jul 06 18:59:20.024135 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.024081    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1195cd91-3196-4506-9a4d-cfe3f673c1e6" containerName="main"
Jul 06 18:59:20.028599 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.028567    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.035624 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.035596    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-47f1eb\"/\"openshift-service-ca.crt\""
Jul 06 18:59:20.035779 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.035714    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-47f1eb\"/\"default-dockercfg-wsdmw\""
Jul 06 18:59:20.037068 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.037046    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-47f1eb\"/\"kube-root-ca.crt\""
Jul 06 18:59:20.039590 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.039556    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-47f1eb\"/\"rate-test-model-47f1eb-kserve-self-signed-certs\""
Jul 06 18:59:20.041324 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.041305    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"]
Jul 06 18:59:20.147319 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.147276    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-dshm\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.147512 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.147357    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b18433-0855-405d-87de-48d7b5b8e14c-tls-certs\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.147512 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.147396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-tmp-dir\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.147512 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.147478    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-home\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.147649 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.147529    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8tcb\" (UniqueName: \"kubernetes.io/projected/04b18433-0855-405d-87de-48d7b5b8e14c-kube-api-access-l8tcb\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.147649 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.147573    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-model-cache\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.248667 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.248626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-home\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.248903 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.248675    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8tcb\" (UniqueName: \"kubernetes.io/projected/04b18433-0855-405d-87de-48d7b5b8e14c-kube-api-access-l8tcb\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.248903 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.248725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-model-cache\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.248903 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.248784    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-dshm\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.248903 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.248825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b18433-0855-405d-87de-48d7b5b8e14c-tls-certs\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.248903 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.248852    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-tmp-dir\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.249184 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.249162    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-home\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.249242 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.249163    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-model-cache\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.249242 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.249213    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-tmp-dir\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.251243 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.251220    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-dshm\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.251415 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.251398    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b18433-0855-405d-87de-48d7b5b8e14c-tls-certs\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.259565 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.259539    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8tcb\" (UniqueName: \"kubernetes.io/projected/04b18433-0855-405d-87de-48d7b5b8e14c-kube-api-access-l8tcb\") pod \"rate-test-model-47f1eb-kserve-78557c4ff5-xt76l\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") " pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.338767 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.338636    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:20.498704 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:20.498640    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"]
Jul 06 18:59:20.504528 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:59:20.504492    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04b18433_0855_405d_87de_48d7b5b8e14c.slice/crio-d5f0f80511f105a5cd076f0acea1c62c7eee23535f5d7ef0d456c7763bb8c5d0 WatchSource:0}: Error finding container d5f0f80511f105a5cd076f0acea1c62c7eee23535f5d7ef0d456c7763bb8c5d0: Status 404 returned error can't find the container with id d5f0f80511f105a5cd076f0acea1c62c7eee23535f5d7ef0d456c7763bb8c5d0
Jul 06 18:59:21.079902 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:21.079866    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l" event={"ID":"04b18433-0855-405d-87de-48d7b5b8e14c","Type":"ContainerStarted","Data":"3cf3509ea9bfb6cb4661de7c160d6a11be50f64f806e10b7eb9d67b4fffe1396"}
Jul 06 18:59:21.079902 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:21.079909    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l" event={"ID":"04b18433-0855-405d-87de-48d7b5b8e14c","Type":"ContainerStarted","Data":"d5f0f80511f105a5cd076f0acea1c62c7eee23535f5d7ef0d456c7763bb8c5d0"}
Jul 06 18:59:21.103238 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:21.103185    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l" podStartSLOduration=2.10316365 podStartE2EDuration="2.10316365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:59:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:21.101483728 +0000 UTC m=+2395.074933241" watchObservedRunningTime="2026-07-06 18:59:21.10316365 +0000 UTC m=+2395.076613213"
Jul 06 18:59:23.573269 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:23.573230    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 18:59:27.565243 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:27.565213    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:59:27.569386 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:27.569359    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nv4mk_34b35881-3a46-4ae5-8e1b-0880314612ec/ovn-acl-logging/0.log"
Jul 06 18:59:30.339040 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:30.338999    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:30.339541 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:30.339058    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:30.351802 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:30.351773    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:31.139132 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:31.139099    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 18:59:57.457042 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.457005    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"]
Jul 06 18:59:57.460405 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.460386    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.463138 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.463109    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b19850\"/\"openshift-service-ca.crt\""
Jul 06 18:59:57.464235 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.464203    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b19850\"/\"kube-root-ca.crt\""
Jul 06 18:59:57.464235 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.464216    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b19850\"/\"rate-test-model-b19850-kserve-self-signed-certs\""
Jul 06 18:59:57.464385 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.464245    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b19850\"/\"default-dockercfg-c67nr\""
Jul 06 18:59:57.478845 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.478809    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"]
Jul 06 18:59:57.561383 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.561339    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt725\" (UniqueName: \"kubernetes.io/projected/2f6138ca-b061-439e-a51d-5b1aa1ff389b-kube-api-access-bt725\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.561383 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.561386    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-tmp-dir\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.561618 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.561404    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-home\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.561618 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.561481    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-dshm\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.561618 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.561524    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-model-cache\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.561618 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.561559    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f6138ca-b061-439e-a51d-5b1aa1ff389b-tls-certs\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.662575 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.662538    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt725\" (UniqueName: \"kubernetes.io/projected/2f6138ca-b061-439e-a51d-5b1aa1ff389b-kube-api-access-bt725\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.662575 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.662578    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-tmp-dir\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.662843 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.662598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-home\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.662843 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.662624    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-dshm\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.662843 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.662645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-model-cache\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.662843 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.662686    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f6138ca-b061-439e-a51d-5b1aa1ff389b-tls-certs\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.663051 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.663030    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-home\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.663108 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.663087    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-model-cache\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.663159 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.663137    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-tmp-dir\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.664985 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.664962    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-dshm\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.665283 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.665263    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f6138ca-b061-439e-a51d-5b1aa1ff389b-tls-certs\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.674775 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.674754    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt725\" (UniqueName: \"kubernetes.io/projected/2f6138ca-b061-439e-a51d-5b1aa1ff389b-kube-api-access-bt725\") pod \"rate-test-model-b19850-kserve-5dcf787fc5-gkwlv\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") " pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.771025 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.770928    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 18:59:57.917965 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:57.917938    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"]
Jul 06 18:59:57.919823 ip-10-0-141-229 kubenswrapper[2581]: W0706 18:59:57.919799    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f6138ca_b061_439e_a51d_5b1aa1ff389b.slice/crio-446d3c1ea55eeef41bb10e5a3e2d415ef4cb135c2977c6b07083cb15715c4914 WatchSource:0}: Error finding container 446d3c1ea55eeef41bb10e5a3e2d415ef4cb135c2977c6b07083cb15715c4914: Status 404 returned error can't find the container with id 446d3c1ea55eeef41bb10e5a3e2d415ef4cb135c2977c6b07083cb15715c4914
Jul 06 18:59:58.255361 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:58.255323    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv" event={"ID":"2f6138ca-b061-439e-a51d-5b1aa1ff389b","Type":"ContainerStarted","Data":"26fa745322ba9dd897fced6dbb72f0e58fd4772df506f561e0d16abc531cc820"}
Jul 06 18:59:58.255361 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:58.255367    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv" event={"ID":"2f6138ca-b061-439e-a51d-5b1aa1ff389b","Type":"ContainerStarted","Data":"446d3c1ea55eeef41bb10e5a3e2d415ef4cb135c2977c6b07083cb15715c4914"}
Jul 06 18:59:58.281255 ip-10-0-141-229 kubenswrapper[2581]: I0706 18:59:58.281199    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv" podStartSLOduration=1.2811805920000001 podStartE2EDuration="1.281180592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:59:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:58.279368792 +0000 UTC m=+2432.252818306" watchObservedRunningTime="2026-07-06 18:59:58.281180592 +0000 UTC m=+2432.254630104"
Jul 06 19:00:00.157783 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.157744    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p"]
Jul 06 19:00:00.160920 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.160898    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p"
Jul 06 19:00:00.164834 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.164812    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-hgx44\""
Jul 06 19:00:00.167016 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.166991    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722740-kp5qv"]
Jul 06 19:00:00.170479 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.170462    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc"]
Jul 06 19:00:00.170622 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.170603    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722740-kp5qv"
Jul 06 19:00:00.173362 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.173349    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc"
Jul 06 19:00:00.181102 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.181079    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p"]
Jul 06 19:00:00.184208 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.184143    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hwk2\" (UniqueName: \"kubernetes.io/projected/26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9-kube-api-access-8hwk2\") pod \"maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p\" (UID: \"26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p"
Jul 06 19:00:00.188653 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.188622    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc"]
Jul 06 19:00:00.192230 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.192195    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722740-kp5qv"]
Jul 06 19:00:00.285162 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.285117    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sb2g\" (UniqueName: \"kubernetes.io/projected/c81053e0-0fe3-49ad-a174-2e3d255fe58c-kube-api-access-5sb2g\") pod \"maas-api-key-cleanup-29722740-kp5qv\" (UID: \"c81053e0-0fe3-49ad-a174-2e3d255fe58c\") " pod="opendatahub/maas-api-key-cleanup-29722740-kp5qv"
Jul 06 19:00:00.285162 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.285169    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss6dx\" (UniqueName: \"kubernetes.io/projected/ba684530-f9ae-4ba3-9016-8db2ca4ebb2f-kube-api-access-ss6dx\") pod \"maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc\" (UID: \"ba684530-f9ae-4ba3-9016-8db2ca4ebb2f\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc"
Jul 06 19:00:00.285441 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.285215    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hwk2\" (UniqueName: \"kubernetes.io/projected/26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9-kube-api-access-8hwk2\") pod \"maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p\" (UID: \"26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p"
Jul 06 19:00:00.300495 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.300459    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hwk2\" (UniqueName: \"kubernetes.io/projected/26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9-kube-api-access-8hwk2\") pod \"maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p\" (UID: \"26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p"
Jul 06 19:00:00.386816 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.386775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sb2g\" (UniqueName: \"kubernetes.io/projected/c81053e0-0fe3-49ad-a174-2e3d255fe58c-kube-api-access-5sb2g\") pod \"maas-api-key-cleanup-29722740-kp5qv\" (UID: \"c81053e0-0fe3-49ad-a174-2e3d255fe58c\") " pod="opendatahub/maas-api-key-cleanup-29722740-kp5qv"
Jul 06 19:00:00.386816 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.386825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss6dx\" (UniqueName: \"kubernetes.io/projected/ba684530-f9ae-4ba3-9016-8db2ca4ebb2f-kube-api-access-ss6dx\") pod \"maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc\" (UID: \"ba684530-f9ae-4ba3-9016-8db2ca4ebb2f\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc"
Jul 06 19:00:00.396751 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.396692    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss6dx\" (UniqueName: \"kubernetes.io/projected/ba684530-f9ae-4ba3-9016-8db2ca4ebb2f-kube-api-access-ss6dx\") pod \"maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc\" (UID: \"ba684530-f9ae-4ba3-9016-8db2ca4ebb2f\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc"
Jul 06 19:00:00.398728 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.398703    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sb2g\" (UniqueName: \"kubernetes.io/projected/c81053e0-0fe3-49ad-a174-2e3d255fe58c-kube-api-access-5sb2g\") pod \"maas-api-key-cleanup-29722740-kp5qv\" (UID: \"c81053e0-0fe3-49ad-a174-2e3d255fe58c\") " pod="opendatahub/maas-api-key-cleanup-29722740-kp5qv"
Jul 06 19:00:00.472280 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.472192    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p"
Jul 06 19:00:00.482540 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.482510    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722740-kp5qv"
Jul 06 19:00:00.491931 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.491901    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc"
Jul 06 19:00:00.644417 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.644392    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p"]
Jul 06 19:00:00.647022 ip-10-0-141-229 kubenswrapper[2581]: W0706 19:00:00.646996    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26b9d3a9_09ce_49f5_a3dd_ca27e30cd0d9.slice/crio-58e4a2b7c70ab0256edaaac57f65f9312fe19bbc724827cd2c104ca6386d16c6 WatchSource:0}: Error finding container 58e4a2b7c70ab0256edaaac57f65f9312fe19bbc724827cd2c104ca6386d16c6: Status 404 returned error can't find the container with id 58e4a2b7c70ab0256edaaac57f65f9312fe19bbc724827cd2c104ca6386d16c6
Jul 06 19:00:00.874584 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.874554    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc"]
Jul 06 19:00:00.876807 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:00.876781    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722740-kp5qv"]
Jul 06 19:00:00.876975 ip-10-0-141-229 kubenswrapper[2581]: W0706 19:00:00.876947    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba684530_f9ae_4ba3_9016_8db2ca4ebb2f.slice/crio-b85cebadfa267f4bfe023bfa8341af6e8a130f933034c4e14979b9e4fe818132 WatchSource:0}: Error finding container b85cebadfa267f4bfe023bfa8341af6e8a130f933034c4e14979b9e4fe818132: Status 404 returned error can't find the container with id b85cebadfa267f4bfe023bfa8341af6e8a130f933034c4e14979b9e4fe818132
Jul 06 19:00:00.879318 ip-10-0-141-229 kubenswrapper[2581]: W0706 19:00:00.879298    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc81053e0_0fe3_49ad_a174_2e3d255fe58c.slice/crio-b6b473cd1bf672c75dd0d7710c0f5058e5eef940119ffb248ba4b9e0f5ab80ff WatchSource:0}: Error finding container b6b473cd1bf672c75dd0d7710c0f5058e5eef940119ffb248ba4b9e0f5ab80ff: Status 404 returned error can't find the container with id b6b473cd1bf672c75dd0d7710c0f5058e5eef940119ffb248ba4b9e0f5ab80ff
Jul 06 19:00:01.270498 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:01.270456    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc" event={"ID":"ba684530-f9ae-4ba3-9016-8db2ca4ebb2f","Type":"ContainerStarted","Data":"b85cebadfa267f4bfe023bfa8341af6e8a130f933034c4e14979b9e4fe818132"}
Jul 06 19:00:01.271985 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:01.271958    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p" event={"ID":"26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9","Type":"ContainerStarted","Data":"f6d2a133d92e64027dbca5a54cc523bf9808b7cc4de5b4613ac39418d0d4b7e8"}
Jul 06 19:00:01.272128 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:01.272006    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p" event={"ID":"26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9","Type":"ContainerStarted","Data":"58e4a2b7c70ab0256edaaac57f65f9312fe19bbc724827cd2c104ca6386d16c6"}
Jul 06 19:00:01.273153 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:01.273135    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722740-kp5qv" event={"ID":"c81053e0-0fe3-49ad-a174-2e3d255fe58c","Type":"ContainerStarted","Data":"b6b473cd1bf672c75dd0d7710c0f5058e5eef940119ffb248ba4b9e0f5ab80ff"}
Jul 06 19:00:01.290665 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:01.290615    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p" podStartSLOduration=1.290598269 podStartE2EDuration="1.290598269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:01.289492499 +0000 UTC m=+2435.262942003" watchObservedRunningTime="2026-07-06 19:00:01.290598269 +0000 UTC m=+2435.264047799"
Jul 06 19:00:01.844780 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:01.844745    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 19:00:02.279869 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:02.279834    2581 generic.go:356] "Generic (PLEG): container finished" podID="26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9" containerID="f6d2a133d92e64027dbca5a54cc523bf9808b7cc4de5b4613ac39418d0d4b7e8" exitCode=0
Jul 06 19:00:02.280280 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:02.279919    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p" event={"ID":"26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9","Type":"ContainerDied","Data":"f6d2a133d92e64027dbca5a54cc523bf9808b7cc4de5b4613ac39418d0d4b7e8"}
Jul 06 19:00:02.281492 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:02.281466    2581 generic.go:356] "Generic (PLEG): container finished" podID="c81053e0-0fe3-49ad-a174-2e3d255fe58c" containerID="a3524ba262e3b592cd71f032185b8ad6cedaae256cbb5ae2094f92f34ce48f6c" exitCode=0
Jul 06 19:00:02.281621 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:02.281545    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722740-kp5qv" event={"ID":"c81053e0-0fe3-49ad-a174-2e3d255fe58c","Type":"ContainerDied","Data":"a3524ba262e3b592cd71f032185b8ad6cedaae256cbb5ae2094f92f34ce48f6c"}
Jul 06 19:00:02.283080 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:02.283046    2581 generic.go:356] "Generic (PLEG): container finished" podID="ba684530-f9ae-4ba3-9016-8db2ca4ebb2f" containerID="e252fdc03c5efa5e133dc17296c651d39d34dcba075f305300c32266514cfd20" exitCode=0
Jul 06 19:00:02.283080 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:02.283071    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc" event={"ID":"ba684530-f9ae-4ba3-9016-8db2ca4ebb2f","Type":"ContainerDied","Data":"e252fdc03c5efa5e133dc17296c651d39d34dcba075f305300c32266514cfd20"}
Jul 06 19:00:03.435964 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:03.435925    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc"
Jul 06 19:00:03.486232 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:03.486199    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p"
Jul 06 19:00:03.489985 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:03.489957    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722740-kp5qv"
Jul 06 19:00:03.514758 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:03.514698    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss6dx\" (UniqueName: \"kubernetes.io/projected/ba684530-f9ae-4ba3-9016-8db2ca4ebb2f-kube-api-access-ss6dx\") pod \"ba684530-f9ae-4ba3-9016-8db2ca4ebb2f\" (UID: \"ba684530-f9ae-4ba3-9016-8db2ca4ebb2f\") "
Jul 06 19:00:03.516906 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:03.516877    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba684530-f9ae-4ba3-9016-8db2ca4ebb2f-kube-api-access-ss6dx" (OuterVolumeSpecName: "kube-api-access-ss6dx") pod "ba684530-f9ae-4ba3-9016-8db2ca4ebb2f" (UID: "ba684530-f9ae-4ba3-9016-8db2ca4ebb2f"). InnerVolumeSpecName "kube-api-access-ss6dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:00:03.615572 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:03.615485    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sb2g\" (UniqueName: \"kubernetes.io/projected/c81053e0-0fe3-49ad-a174-2e3d255fe58c-kube-api-access-5sb2g\") pod \"c81053e0-0fe3-49ad-a174-2e3d255fe58c\" (UID: \"c81053e0-0fe3-49ad-a174-2e3d255fe58c\") "
Jul 06 19:00:03.615572 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:03.615550    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hwk2\" (UniqueName: \"kubernetes.io/projected/26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9-kube-api-access-8hwk2\") pod \"26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9\" (UID: \"26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9\") "
Jul 06 19:00:03.615859 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:03.615764    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss6dx\" (UniqueName: \"kubernetes.io/projected/ba684530-f9ae-4ba3-9016-8db2ca4ebb2f-kube-api-access-ss6dx\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 19:00:03.617567 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:03.617538    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c81053e0-0fe3-49ad-a174-2e3d255fe58c-kube-api-access-5sb2g" (OuterVolumeSpecName: "kube-api-access-5sb2g") pod "c81053e0-0fe3-49ad-a174-2e3d255fe58c" (UID: "c81053e0-0fe3-49ad-a174-2e3d255fe58c"). InnerVolumeSpecName "kube-api-access-5sb2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:00:03.617652 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:03.617545    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9-kube-api-access-8hwk2" (OuterVolumeSpecName: "kube-api-access-8hwk2") pod "26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9" (UID: "26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9"). InnerVolumeSpecName "kube-api-access-8hwk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:00:03.716901 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:03.716860    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hwk2\" (UniqueName: \"kubernetes.io/projected/26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9-kube-api-access-8hwk2\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 19:00:03.716901 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:03.716894    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sb2g\" (UniqueName: \"kubernetes.io/projected/c81053e0-0fe3-49ad-a174-2e3d255fe58c-kube-api-access-5sb2g\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 19:00:04.295380 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:04.295349    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p"
Jul 06 19:00:04.295581 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:04.295352    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p" event={"ID":"26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9","Type":"ContainerDied","Data":"58e4a2b7c70ab0256edaaac57f65f9312fe19bbc724827cd2c104ca6386d16c6"}
Jul 06 19:00:04.295581 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:04.295494    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58e4a2b7c70ab0256edaaac57f65f9312fe19bbc724827cd2c104ca6386d16c6"
Jul 06 19:00:04.296795 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:04.296773    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722740-kp5qv"
Jul 06 19:00:04.296935 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:04.296772    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722740-kp5qv" event={"ID":"c81053e0-0fe3-49ad-a174-2e3d255fe58c","Type":"ContainerDied","Data":"b6b473cd1bf672c75dd0d7710c0f5058e5eef940119ffb248ba4b9e0f5ab80ff"}
Jul 06 19:00:04.296935 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:04.296905    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b6b473cd1bf672c75dd0d7710c0f5058e5eef940119ffb248ba4b9e0f5ab80ff"
Jul 06 19:00:04.298093 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:04.298073    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc"
Jul 06 19:00:04.298194 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:04.298092    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc" event={"ID":"ba684530-f9ae-4ba3-9016-8db2ca4ebb2f","Type":"ContainerDied","Data":"b85cebadfa267f4bfe023bfa8341af6e8a130f933034c4e14979b9e4fe818132"}
Jul 06 19:00:04.298194 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:04.298129    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b85cebadfa267f4bfe023bfa8341af6e8a130f933034c4e14979b9e4fe818132"
Jul 06 19:00:07.771571 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:07.771536    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 19:00:07.772007 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:07.771599    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 19:00:07.784496 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:07.784468    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 19:00:08.329204 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:08.329173    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 19:00:16.367712 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:16.367680    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 19:00:31.284275 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:31.284238    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 19:00:37.368439 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:37.368405    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"]
Jul 06 19:00:38.378322 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:38.378271    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"]
Jul 06 19:00:45.083530 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:45.083477    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 19:00:52.388379 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.388320    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l" podUID="04b18433-0855-405d-87de-48d7b5b8e14c" containerName="main" containerID="cri-o://3cf3509ea9bfb6cb4661de7c160d6a11be50f64f806e10b7eb9d67b4fffe1396" gracePeriod=45
Jul 06 19:00:52.542901 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.542866    2581 generic.go:356] "Generic (PLEG): container finished" podID="04b18433-0855-405d-87de-48d7b5b8e14c" containerID="3cf3509ea9bfb6cb4661de7c160d6a11be50f64f806e10b7eb9d67b4fffe1396" exitCode=0
Jul 06 19:00:52.543061 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.542943    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l" event={"ID":"04b18433-0855-405d-87de-48d7b5b8e14c","Type":"ContainerDied","Data":"3cf3509ea9bfb6cb4661de7c160d6a11be50f64f806e10b7eb9d67b4fffe1396"}
Jul 06 19:00:52.639883 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.639825    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 19:00:52.752525 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.752479    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-tmp-dir\") pod \"04b18433-0855-405d-87de-48d7b5b8e14c\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") "
Jul 06 19:00:52.752525 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.752530    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b18433-0855-405d-87de-48d7b5b8e14c-tls-certs\") pod \"04b18433-0855-405d-87de-48d7b5b8e14c\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") "
Jul 06 19:00:52.752818 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.752545    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-model-cache\") pod \"04b18433-0855-405d-87de-48d7b5b8e14c\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") "
Jul 06 19:00:52.752818 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.752563    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-home\") pod \"04b18433-0855-405d-87de-48d7b5b8e14c\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") "
Jul 06 19:00:52.752818 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.752616    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8tcb\" (UniqueName: \"kubernetes.io/projected/04b18433-0855-405d-87de-48d7b5b8e14c-kube-api-access-l8tcb\") pod \"04b18433-0855-405d-87de-48d7b5b8e14c\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") "
Jul 06 19:00:52.752818 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.752653    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-dshm\") pod \"04b18433-0855-405d-87de-48d7b5b8e14c\" (UID: \"04b18433-0855-405d-87de-48d7b5b8e14c\") "
Jul 06 19:00:52.752818 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.752804    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "04b18433-0855-405d-87de-48d7b5b8e14c" (UID: "04b18433-0855-405d-87de-48d7b5b8e14c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:00:52.753201 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.752820    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-model-cache" (OuterVolumeSpecName: "model-cache") pod "04b18433-0855-405d-87de-48d7b5b8e14c" (UID: "04b18433-0855-405d-87de-48d7b5b8e14c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:00:52.753201 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.752925    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-home" (OuterVolumeSpecName: "home") pod "04b18433-0855-405d-87de-48d7b5b8e14c" (UID: "04b18433-0855-405d-87de-48d7b5b8e14c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:00:52.753201 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.753010    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-tmp-dir\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 19:00:52.753201 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.753031    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-model-cache\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 19:00:52.754694 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.754663    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04b18433-0855-405d-87de-48d7b5b8e14c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "04b18433-0855-405d-87de-48d7b5b8e14c" (UID: "04b18433-0855-405d-87de-48d7b5b8e14c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:00:52.754694 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.754651    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-dshm" (OuterVolumeSpecName: "dshm") pod "04b18433-0855-405d-87de-48d7b5b8e14c" (UID: "04b18433-0855-405d-87de-48d7b5b8e14c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:00:52.754859 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.754766    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b18433-0855-405d-87de-48d7b5b8e14c-kube-api-access-l8tcb" (OuterVolumeSpecName: "kube-api-access-l8tcb") pod "04b18433-0855-405d-87de-48d7b5b8e14c" (UID: "04b18433-0855-405d-87de-48d7b5b8e14c"). InnerVolumeSpecName "kube-api-access-l8tcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:00:52.854088 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.854047    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b18433-0855-405d-87de-48d7b5b8e14c-tls-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 19:00:52.854088 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.854083    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-home\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 19:00:52.854088 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.854093    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8tcb\" (UniqueName: \"kubernetes.io/projected/04b18433-0855-405d-87de-48d7b5b8e14c-kube-api-access-l8tcb\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 19:00:52.854088 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:52.854102    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b18433-0855-405d-87de-48d7b5b8e14c-dshm\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 19:00:53.404169 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.404129    2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv" podUID="2f6138ca-b061-439e-a51d-5b1aa1ff389b" containerName="main" containerID="cri-o://26fa745322ba9dd897fced6dbb72f0e58fd4772df506f561e0d16abc531cc820" gracePeriod=45
Jul 06 19:00:53.550236 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.550198    2581 generic.go:356] "Generic (PLEG): container finished" podID="2f6138ca-b061-439e-a51d-5b1aa1ff389b" containerID="26fa745322ba9dd897fced6dbb72f0e58fd4772df506f561e0d16abc531cc820" exitCode=0
Jul 06 19:00:53.550434 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.550270    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv" event={"ID":"2f6138ca-b061-439e-a51d-5b1aa1ff389b","Type":"ContainerDied","Data":"26fa745322ba9dd897fced6dbb72f0e58fd4772df506f561e0d16abc531cc820"}
Jul 06 19:00:53.551716 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.551693    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"
Jul 06 19:00:53.551879 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.551723    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l" event={"ID":"04b18433-0855-405d-87de-48d7b5b8e14c","Type":"ContainerDied","Data":"d5f0f80511f105a5cd076f0acea1c62c7eee23535f5d7ef0d456c7763bb8c5d0"}
Jul 06 19:00:53.551879 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.551786    2581 scope.go:117] "RemoveContainer" containerID="3cf3509ea9bfb6cb4661de7c160d6a11be50f64f806e10b7eb9d67b4fffe1396"
Jul 06 19:00:53.574045 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.574018    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"]
Jul 06 19:00:53.578546 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.578250    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-47f1eb/rate-test-model-47f1eb-kserve-78557c4ff5-xt76l"]
Jul 06 19:00:53.662181 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.662113    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 19:00:53.761927 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.761891    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-tmp-dir\") pod \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") "
Jul 06 19:00:53.761927 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.761931    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-model-cache\") pod \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") "
Jul 06 19:00:53.762308 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.761957    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-home\") pod \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") "
Jul 06 19:00:53.762308 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.761987    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-dshm\") pod \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") "
Jul 06 19:00:53.762308 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.762022    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f6138ca-b061-439e-a51d-5b1aa1ff389b-tls-certs\") pod \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") "
Jul 06 19:00:53.762308 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.762088    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bt725\" (UniqueName: \"kubernetes.io/projected/2f6138ca-b061-439e-a51d-5b1aa1ff389b-kube-api-access-bt725\") pod \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\" (UID: \"2f6138ca-b061-439e-a51d-5b1aa1ff389b\") "
Jul 06 19:00:53.762308 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.762197    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-model-cache" (OuterVolumeSpecName: "model-cache") pod "2f6138ca-b061-439e-a51d-5b1aa1ff389b" (UID: "2f6138ca-b061-439e-a51d-5b1aa1ff389b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:00:53.762308 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.762221    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2f6138ca-b061-439e-a51d-5b1aa1ff389b" (UID: "2f6138ca-b061-439e-a51d-5b1aa1ff389b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:00:53.762308 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.762239    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-home" (OuterVolumeSpecName: "home") pod "2f6138ca-b061-439e-a51d-5b1aa1ff389b" (UID: "2f6138ca-b061-439e-a51d-5b1aa1ff389b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:00:53.762660 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.762322    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-tmp-dir\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 19:00:53.762660 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.762338    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-model-cache\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 19:00:53.762660 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.762349    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-home\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 19:00:53.764161 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.764133    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-dshm" (OuterVolumeSpecName: "dshm") pod "2f6138ca-b061-439e-a51d-5b1aa1ff389b" (UID: "2f6138ca-b061-439e-a51d-5b1aa1ff389b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:00:53.764352 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.764332    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f6138ca-b061-439e-a51d-5b1aa1ff389b-kube-api-access-bt725" (OuterVolumeSpecName: "kube-api-access-bt725") pod "2f6138ca-b061-439e-a51d-5b1aa1ff389b" (UID: "2f6138ca-b061-439e-a51d-5b1aa1ff389b"). InnerVolumeSpecName "kube-api-access-bt725". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:00:53.764400 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.764350    2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f6138ca-b061-439e-a51d-5b1aa1ff389b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2f6138ca-b061-439e-a51d-5b1aa1ff389b" (UID: "2f6138ca-b061-439e-a51d-5b1aa1ff389b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:00:53.863655 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.863612    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f6138ca-b061-439e-a51d-5b1aa1ff389b-dshm\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 19:00:53.863865 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.863668    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f6138ca-b061-439e-a51d-5b1aa1ff389b-tls-certs\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 19:00:53.863865 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:53.863680    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bt725\" (UniqueName: \"kubernetes.io/projected/2f6138ca-b061-439e-a51d-5b1aa1ff389b-kube-api-access-bt725\") on node \"ip-10-0-141-229.ec2.internal\" DevicePath \"\""
Jul 06 19:00:54.558519 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:54.558483    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv" event={"ID":"2f6138ca-b061-439e-a51d-5b1aa1ff389b","Type":"ContainerDied","Data":"446d3c1ea55eeef41bb10e5a3e2d415ef4cb135c2977c6b07083cb15715c4914"}
Jul 06 19:00:54.558962 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:54.558533    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"
Jul 06 19:00:54.558962 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:54.558536    2581 scope.go:117] "RemoveContainer" containerID="26fa745322ba9dd897fced6dbb72f0e58fd4772df506f561e0d16abc531cc820"
Jul 06 19:00:54.580809 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:54.580781    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"]
Jul 06 19:00:54.587837 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:54.587808    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b19850/rate-test-model-b19850-kserve-5dcf787fc5-gkwlv"]
Jul 06 19:00:55.471433 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:55.471390    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04b18433-0855-405d-87de-48d7b5b8e14c" path="/var/lib/kubelet/pods/04b18433-0855-405d-87de-48d7b5b8e14c/volumes"
Jul 06 19:00:55.471926 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:00:55.471903    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f6138ca-b061-439e-a51d-5b1aa1ff389b" path="/var/lib/kubelet/pods/2f6138ca-b061-439e-a51d-5b1aa1ff389b/volumes"
Jul 06 19:01:48.550327 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:01:48.550296    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p"]
Jul 06 19:01:48.558778 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:01:48.558748    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-47f1eb-29722740-dr84p"]
Jul 06 19:01:49.470908 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:01:49.470876    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9" path="/var/lib/kubelet/pods/26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9/volumes"
Jul 06 19:01:51.790183 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:01:51.790144    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 19:01:57.452510 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:01:57.452474    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc"]
Jul 06 19:01:57.462807 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:01:57.462765    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-b19850-29722740-gdrgc"]
Jul 06 19:01:57.472593 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:01:57.472559    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba684530-f9ae-4ba3-9016-8db2ca4ebb2f" path="/var/lib/kubelet/pods/ba684530-f9ae-4ba3-9016-8db2ca4ebb2f/volumes"
Jul 06 19:02:04.990483 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:04.990445    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d8lz6"]
Jul 06 19:02:10.777565 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:10.777532    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wtpwf_0de3db7b-9518-43da-aa7c-31684a8bf481/manager/0.log"
Jul 06 19:02:10.930786 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:10.930756    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-db4pf_69a9328b-5719-4081-ad43-de9724904356/manager/0.log"
Jul 06 19:02:11.255930 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:11.255899    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722710-zptrv_c7b32498-7ed8-4ab1-b35b-3f7af4ea37de/cleanup/0.log"
Jul 06 19:02:11.504773 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:11.504705    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722740-kp5qv_c81053e0-0fe3-49ad-a174-2e3d255fe58c/cleanup/0.log"
Jul 06 19:02:11.808688 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:11.808664    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-zzkcz_c5774162-a816-4352-b3c7-01d7da9fc5e7/server/0.log"
Jul 06 19:02:11.937580 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:11.937548    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ghrjn_56afe3f7-e3c1-4306-a07d-d8e2943ab2c6/manager/0.log"
Jul 06 19:02:12.060794 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:12.060696    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bf95b557-574t4_306ee2d9-ff73-4216-9768-d718aee079a2/copy-manifests/0.log"
Jul 06 19:02:12.089855 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:12.089825    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bf95b557-574t4_306ee2d9-ff73-4216-9768-d718aee079a2/manager/0.log"
Jul 06 19:02:14.203559 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:14.203527    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-dbp8j_da06585e-5e22-4597-bfb2-d2cbfbab9b16/manager/0.log"
Jul 06 19:02:14.610900 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:14.610809    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-gwwxk_a49361a1-af70-4f8e-96a1-8bb0165c0dfd/manager/0.log"
Jul 06 19:02:14.726715 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:14.726685    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-d8lz6_76a5fa6a-1679-49b2-a189-7a80b64d7248/limitador/0.log"
Jul 06 19:02:15.344550 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:15.344517    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-h97l9_69977b49-a4e5-4f49-9b3d-efcd11d53c19/discovery/0.log"
Jul 06 19:02:15.576509 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:15.576480    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-88568c598-wdxnb_de54f326-9c7e-4306-8d7c-d2c463dbed5f/kube-auth-proxy/0.log"
Jul 06 19:02:15.976846 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:15.976820    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-xgbrn_ffc659f1-d433-4a93-83f5-dc4601f49a6f/payload-processing/0.log"
Jul 06 19:02:16.462156 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:16.462120    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-88gtz_fcbd6d15-7dcc-46d0-92b5-62fd6f887d4c/main/0.log"
Jul 06 19:02:16.936165 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:16.936130    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-qc74q_f2968477-c746-42f4-baf0-a3a82444982f/main/0.log"
Jul 06 19:02:25.172019 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:25.171987    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zdtg9_3f857b5c-ed1c-42a0-8960-524770d6f07e/global-pull-secret-syncer/0.log"
Jul 06 19:02:25.346873 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:25.346812    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rqp4s_9d4a8d7b-f63e-46cc-b99b-fe9b655c3388/konnectivity-agent/0.log"
Jul 06 19:02:25.452729 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:25.452637    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-229.ec2.internal_dd0c9e68fa1554fd01092c3bef64ba0f/haproxy/0.log"
Jul 06 19:02:30.782854 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:30.782827    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-dbp8j_da06585e-5e22-4597-bfb2-d2cbfbab9b16/manager/0.log"
Jul 06 19:02:31.066527 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:31.066436    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-gwwxk_a49361a1-af70-4f8e-96a1-8bb0165c0dfd/manager/0.log"
Jul 06 19:02:31.136829 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:31.136772    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-d8lz6_76a5fa6a-1679-49b2-a189-7a80b64d7248/limitador/0.log"
Jul 06 19:02:33.849116 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:33.849079    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h5f5c_1ca034ed-9fa2-4244-921f-29e1aa2b7a53/node-exporter/0.log"
Jul 06 19:02:33.882615 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:33.882580    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h5f5c_1ca034ed-9fa2-4244-921f-29e1aa2b7a53/kube-rbac-proxy/0.log"
Jul 06 19:02:33.913914 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:33.913885    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h5f5c_1ca034ed-9fa2-4244-921f-29e1aa2b7a53/init-textfile/0.log"
Jul 06 19:02:35.248561 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.248526    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"]
Jul 06 19:02:35.248998 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.248893    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f6138ca-b061-439e-a51d-5b1aa1ff389b" containerName="main"
Jul 06 19:02:35.248998 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.248905    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f6138ca-b061-439e-a51d-5b1aa1ff389b" containerName="main"
Jul 06 19:02:35.248998 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.248912    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba684530-f9ae-4ba3-9016-8db2ca4ebb2f" containerName="cleanup"
Jul 06 19:02:35.248998 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.248918    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba684530-f9ae-4ba3-9016-8db2ca4ebb2f" containerName="cleanup"
Jul 06 19:02:35.248998 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.248927    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9" containerName="cleanup"
Jul 06 19:02:35.248998 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.248932    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9" containerName="cleanup"
Jul 06 19:02:35.248998 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.248941    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b18433-0855-405d-87de-48d7b5b8e14c" containerName="main"
Jul 06 19:02:35.248998 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.248946    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b18433-0855-405d-87de-48d7b5b8e14c" containerName="main"
Jul 06 19:02:35.248998 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.248955    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c81053e0-0fe3-49ad-a174-2e3d255fe58c" containerName="cleanup"
Jul 06 19:02:35.248998 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.248960    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c81053e0-0fe3-49ad-a174-2e3d255fe58c" containerName="cleanup"
Jul 06 19:02:35.249292 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.249012    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c81053e0-0fe3-49ad-a174-2e3d255fe58c" containerName="cleanup"
Jul 06 19:02:35.249292 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.249020    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="04b18433-0855-405d-87de-48d7b5b8e14c" containerName="main"
Jul 06 19:02:35.249292 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.249027    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f6138ca-b061-439e-a51d-5b1aa1ff389b" containerName="main"
Jul 06 19:02:35.249292 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.249033    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba684530-f9ae-4ba3-9016-8db2ca4ebb2f" containerName="cleanup"
Jul 06 19:02:35.249292 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.249039    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="26b9d3a9-09ce-49f5-a3dd-ca27e30cd0d9" containerName="cleanup"
Jul 06 19:02:35.251940 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.251920    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.254885 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.254865    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cgk74\"/\"kube-root-ca.crt\""
Jul 06 19:02:35.256128 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.256110    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cgk74\"/\"default-dockercfg-tpbrd\""
Jul 06 19:02:35.256214 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.256130    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cgk74\"/\"openshift-service-ca.crt\""
Jul 06 19:02:35.263914 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.263892    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"]
Jul 06 19:02:35.312244 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.312214    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/422d8b75-945b-458c-a8fc-96a7740c51a5-lib-modules\") pod \"perf-node-gather-daemonset-lt8kp\" (UID: \"422d8b75-945b-458c-a8fc-96a7740c51a5\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.312244 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.312246    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/422d8b75-945b-458c-a8fc-96a7740c51a5-podres\") pod \"perf-node-gather-daemonset-lt8kp\" (UID: \"422d8b75-945b-458c-a8fc-96a7740c51a5\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.312506 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.312262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/422d8b75-945b-458c-a8fc-96a7740c51a5-proc\") pod \"perf-node-gather-daemonset-lt8kp\" (UID: \"422d8b75-945b-458c-a8fc-96a7740c51a5\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.312506 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.312381    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rch6s\" (UniqueName: \"kubernetes.io/projected/422d8b75-945b-458c-a8fc-96a7740c51a5-kube-api-access-rch6s\") pod \"perf-node-gather-daemonset-lt8kp\" (UID: \"422d8b75-945b-458c-a8fc-96a7740c51a5\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.312506 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.312420    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/422d8b75-945b-458c-a8fc-96a7740c51a5-sys\") pod \"perf-node-gather-daemonset-lt8kp\" (UID: \"422d8b75-945b-458c-a8fc-96a7740c51a5\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.413724 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.413680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rch6s\" (UniqueName: \"kubernetes.io/projected/422d8b75-945b-458c-a8fc-96a7740c51a5-kube-api-access-rch6s\") pod \"perf-node-gather-daemonset-lt8kp\" (UID: \"422d8b75-945b-458c-a8fc-96a7740c51a5\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.413724 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.413728    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/422d8b75-945b-458c-a8fc-96a7740c51a5-sys\") pod \"perf-node-gather-daemonset-lt8kp\" (UID: \"422d8b75-945b-458c-a8fc-96a7740c51a5\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.413967 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.413785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/422d8b75-945b-458c-a8fc-96a7740c51a5-lib-modules\") pod \"perf-node-gather-daemonset-lt8kp\" (UID: \"422d8b75-945b-458c-a8fc-96a7740c51a5\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.413967 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.413802    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/422d8b75-945b-458c-a8fc-96a7740c51a5-podres\") pod \"perf-node-gather-daemonset-lt8kp\" (UID: \"422d8b75-945b-458c-a8fc-96a7740c51a5\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.413967 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.413818    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/422d8b75-945b-458c-a8fc-96a7740c51a5-proc\") pod \"perf-node-gather-daemonset-lt8kp\" (UID: \"422d8b75-945b-458c-a8fc-96a7740c51a5\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.413967 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.413928    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/422d8b75-945b-458c-a8fc-96a7740c51a5-sys\") pod \"perf-node-gather-daemonset-lt8kp\" (UID: \"422d8b75-945b-458c-a8fc-96a7740c51a5\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.413967 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.413951    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/422d8b75-945b-458c-a8fc-96a7740c51a5-podres\") pod \"perf-node-gather-daemonset-lt8kp\" (UID: \"422d8b75-945b-458c-a8fc-96a7740c51a5\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.414142 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.413969    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/422d8b75-945b-458c-a8fc-96a7740c51a5-lib-modules\") pod \"perf-node-gather-daemonset-lt8kp\" (UID: \"422d8b75-945b-458c-a8fc-96a7740c51a5\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.414142 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.413993    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/422d8b75-945b-458c-a8fc-96a7740c51a5-proc\") pod \"perf-node-gather-daemonset-lt8kp\" (UID: \"422d8b75-945b-458c-a8fc-96a7740c51a5\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.426019 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.425993    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rch6s\" (UniqueName: \"kubernetes.io/projected/422d8b75-945b-458c-a8fc-96a7740c51a5-kube-api-access-rch6s\") pod \"perf-node-gather-daemonset-lt8kp\" (UID: \"422d8b75-945b-458c-a8fc-96a7740c51a5\") " pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.561656 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.561567    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:35.698060 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:35.697985    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"]
Jul 06 19:02:35.700359 ip-10-0-141-229 kubenswrapper[2581]: W0706 19:02:35.700326    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod422d8b75_945b_458c_a8fc_96a7740c51a5.slice/crio-0c8ceb2a249667d55752d098e3b99f2cb72a77b9e3b8fb52b3bd6a1b44012a5d WatchSource:0}: Error finding container 0c8ceb2a249667d55752d098e3b99f2cb72a77b9e3b8fb52b3bd6a1b44012a5d: Status 404 returned error can't find the container with id 0c8ceb2a249667d55752d098e3b99f2cb72a77b9e3b8fb52b3bd6a1b44012a5d
Jul 06 19:02:36.024574 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:36.024532    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp" event={"ID":"422d8b75-945b-458c-a8fc-96a7740c51a5","Type":"ContainerStarted","Data":"d12328d9e03ff9976cbfe201bc2e050e23fb091c73bc8c2af6fe118e3466cad7"}
Jul 06 19:02:36.024764 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:36.024597    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp" event={"ID":"422d8b75-945b-458c-a8fc-96a7740c51a5","Type":"ContainerStarted","Data":"0c8ceb2a249667d55752d098e3b99f2cb72a77b9e3b8fb52b3bd6a1b44012a5d"}
Jul 06 19:02:36.024764 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:36.024675    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:36.047634 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:36.047572    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp" podStartSLOduration=1.047550135 podStartE2EDuration="1.047550135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:02:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:02:36.045256891 +0000 UTC m=+2590.018706399" watchObservedRunningTime="2026-07-06 19:02:36.047550135 +0000 UTC m=+2590.020999704"
Jul 06 19:02:39.209084 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:39.209055    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tlpx5_9c0a0ab6-71e4-48b2-ab8c-d0540b08763d/dns/0.log"
Jul 06 19:02:39.237882 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:39.237851    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tlpx5_9c0a0ab6-71e4-48b2-ab8c-d0540b08763d/kube-rbac-proxy/0.log"
Jul 06 19:02:39.309190 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:39.309158    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mr54p_53792e6a-1284-4ac6-8370-bb836200eb67/dns-node-resolver/0.log"
Jul 06 19:02:40.092190 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:40.092153    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s2ktj_02f96065-7946-46c4-ad55-a5a5a0fc6520/node-ca/0.log"
Jul 06 19:02:41.404955 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:41.404918    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-h97l9_69977b49-a4e5-4f49-9b3d-efcd11d53c19/discovery/0.log"
Jul 06 19:02:41.457823 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:41.457792    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-88568c598-wdxnb_de54f326-9c7e-4306-8d7c-d2c463dbed5f/kube-auth-proxy/0.log"
Jul 06 19:02:41.644562 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:41.644513    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-xgbrn_ffc659f1-d433-4a93-83f5-dc4601f49a6f/payload-processing/0.log"
Jul 06 19:02:42.039312 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:42.039286    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cgk74/perf-node-gather-daemonset-lt8kp"
Jul 06 19:02:42.452586 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:42.452541    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5pcwz_1c696411-a9e1-4f30-ace4-f468430c3bdf/serve-healthcheck-canary/0.log"
Jul 06 19:02:43.068079 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:43.068051    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4kj2l_eb45813e-f621-4f4c-b828-2b21758b7126/kube-rbac-proxy/0.log"
Jul 06 19:02:43.108766 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:43.108713    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4kj2l_eb45813e-f621-4f4c-b828-2b21758b7126/exporter/0.log"
Jul 06 19:02:43.156359 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:43.156334    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4kj2l_eb45813e-f621-4f4c-b828-2b21758b7126/extractor/0.log"
Jul 06 19:02:46.048095 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:46.048065    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wtpwf_0de3db7b-9518-43da-aa7c-31684a8bf481/manager/0.log"
Jul 06 19:02:46.266407 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:46.266375    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-db4pf_69a9328b-5719-4081-ad43-de9724904356/manager/0.log"
Jul 06 19:02:46.504564 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:46.504534    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722710-zptrv_c7b32498-7ed8-4ab1-b35b-3f7af4ea37de/cleanup/0.log"
Jul 06 19:02:46.515457 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:46.515431    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-h97l9_69977b49-a4e5-4f49-9b3d-efcd11d53c19/discovery/0.log"
Jul 06 19:02:46.572496 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:46.572413    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-88568c598-wdxnb_de54f326-9c7e-4306-8d7c-d2c463dbed5f/kube-auth-proxy/0.log"
Jul 06 19:02:46.594237 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:46.594207    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722740-kp5qv_c81053e0-0fe3-49ad-a174-2e3d255fe58c/cleanup/0.log"
Jul 06 19:02:46.746446 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:46.746410    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-xgbrn_ffc659f1-d433-4a93-83f5-dc4601f49a6f/payload-processing/0.log"
Jul 06 19:02:46.775023 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:46.774999    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-zzkcz_c5774162-a816-4352-b3c7-01d7da9fc5e7/server/0.log"
Jul 06 19:02:46.897757 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:46.897702    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ghrjn_56afe3f7-e3c1-4306-a07d-d8e2943ab2c6/manager/0.log"
Jul 06 19:02:47.012859 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:47.012828    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bf95b557-574t4_306ee2d9-ff73-4216-9768-d718aee079a2/manager/0.log"
Jul 06 19:02:47.084315 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:47.084283    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bf95b557-574t4_306ee2d9-ff73-4216-9768-d718aee079a2/copy-manifests/0.log"
Jul 06 19:02:48.257202 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:48.257117    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-h97l9_69977b49-a4e5-4f49-9b3d-efcd11d53c19/discovery/0.log"
Jul 06 19:02:48.455931 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:48.455903    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-88568c598-wdxnb_de54f326-9c7e-4306-8d7c-d2c463dbed5f/kube-auth-proxy/0.log"
Jul 06 19:02:48.685178 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:48.685137    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-xgbrn_ffc659f1-d433-4a93-83f5-dc4601f49a6f/payload-processing/0.log"
Jul 06 19:02:48.940922 ip-10-0-141-229 kubenswrapper[2581]: I0706 19:02:48.940842    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-ftdgm_99b0df35-7d84-45be-9bf2-c7bcc6ffc225/openshift-lws-operator/0.log"