Jul 07 14:26:24.430585 ip-10-0-129-1 systemd[1]: Starting Kubernetes Kubelet...
Jul 07 14:26:24.858209 ip-10-0-129-1 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 14:26:24.858209 ip-10-0-129-1 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 07 14:26:24.858209 ip-10-0-129-1 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 14:26:24.858209 ip-10-0-129-1 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 07 14:26:24.858209 ip-10-0-129-1 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 07 14:26:24.858209 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.856577    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.863808    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.863955    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.863961    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.863964    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.863968    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.863972    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.863974    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.863978    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.863980    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.863983    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.863987    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.863990    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.863993    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.863996    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.863999    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864004    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864007    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864011    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:26:24.920854 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864014    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864026    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864033    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864036    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864040    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864042    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864045    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864048    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864050    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864054    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864058    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864062    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864065    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864068    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864071    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864073    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864076    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864079    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864081    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864084    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:26:25.165944 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864086    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:26:24.941821 ip-10-0-129-1 systemd[1]: Started Kubernetes Kubelet.
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864089    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864092    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864094    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864097    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864100    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864103    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864106    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864108    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864111    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864113    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864116    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864118    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864121    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864124    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864127    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864129    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864132    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864134    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:26:25.178867 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864137    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864139    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864142    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864144    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864147    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864149    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864152    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864154    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864157    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864159    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864162    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864164    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864167    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864169    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864172    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864177    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864181    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864187    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864190    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:26:25.179855 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864194    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864196    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864200    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864203    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864206    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864209    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864212    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864215    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864218    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864221    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864223    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864226    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864230    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864232    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864236    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864239    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864242    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864245    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864247    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864250    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:26:25.180624 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864253    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864256    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.864258    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865024    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865032    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865036    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865039    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865042    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865045    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865048    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865051    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865053    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865056    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865059    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865061    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865064    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865067    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865069    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865072    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865074    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:26:25.181248 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865077    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865080    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865083    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865085    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865087    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865090    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865093    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865096    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865099    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865102    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865104    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865107    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865110    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865113    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865116    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865119    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865121    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865124    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865126    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865129    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:26:25.182091 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865131    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865134    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865137    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865139    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865142    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865144    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865147    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865150    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865152    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865155    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865159    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865163    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865165    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865169    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865171    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865174    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865177    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865179    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865183    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865186    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:26:25.182715 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865189    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865192    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865195    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865197    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865201    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865204    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865207    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865210    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865213    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865216    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865219    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865221    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865224    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865226    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865229    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865231    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865234    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865236    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865239    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:26:25.183748 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865241    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865244    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865248    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865251    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865253    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865256    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865258    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865262    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865264    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865267    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865269    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865295    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865298    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865301    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865303    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865306    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865309    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865311    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865314    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:26:25.184564 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865320    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865323    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865325    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865328    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865719    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865724    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865726    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865729    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865731    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865734    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865737    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865739    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865742    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865745    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865749    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865753    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865756    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865759    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865762    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865765    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:26:25.185298 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865768    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865772    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865774    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865777    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865779    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865782    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865785    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865787    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865790    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865792    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865795    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865798    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865802    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865805    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865807    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865810    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865812    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865814    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865817    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865819    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:26:25.186018 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865822    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865824    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865826    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865829    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865831    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865834    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865837    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865840    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865842    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865845    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865847    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865850    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865852    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865856    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865860    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865864    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865866    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865869    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865872    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:26:25.186682 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865875    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865877    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865880    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865882    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865885    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865889    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865891    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865894    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865897    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865900    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865902    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865905    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865907    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865910    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865913    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865917    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865920    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865923    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865925    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:26:25.187232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865928    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865930    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865935    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865938    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865940    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865943    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865945    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865948    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865951    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865953    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865956    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865958    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865961    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865963    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865966    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865968    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865971    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865973    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865977    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865980    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:26:25.187789 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865982    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865985    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865987    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865990    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.865992    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866067    2583 flags.go:64] FLAG: --address="0.0.0.0"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866073    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866080    2583 flags.go:64] FLAG: --anonymous-auth="true"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866085    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866089    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866091    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866096    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866101    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866104    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866109    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866112    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866115    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866118    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866121    2583 flags.go:64] FLAG: --cgroup-root=""
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866124    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866127    2583 flags.go:64] FLAG: --client-ca-file=""
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866131    2583 flags.go:64] FLAG: --cloud-provider="external"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866134    2583 flags.go:64] FLAG: --cluster-dns="[]"
Jul 07 14:26:25.188401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866138    2583 flags.go:64] FLAG: --cluster-domain=""
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866140    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866144    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866147    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866150    2583 flags.go:64] FLAG: --container-log-max-files="5"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866154    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866157    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866161    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866164    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866167    2583 flags.go:64] FLAG: --contention-profiling="false"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866170    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866173    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866176    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866179    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866184    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866187    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866190    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866193    2583 flags.go:64] FLAG: --enable-load-reader="false"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866196    2583 flags.go:64] FLAG: --enable-server="true"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866198    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866203    2583 flags.go:64] FLAG: --event-burst="100"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866206    2583 flags.go:64] FLAG: --event-qps="50"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866209    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866213    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 07 14:26:25.189031 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866217    2583 flags.go:64] FLAG: --eviction-hard=""
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866221    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866224    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866227    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866230    2583 flags.go:64] FLAG: --eviction-soft=""
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866233    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866235    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866239    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866242    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866245    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866248    2583 flags.go:64] FLAG: --fail-swap-on="true"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866250    2583 flags.go:64] FLAG: --feature-gates=""
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866254    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866257    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866260    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866265    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866268    2583 flags.go:64] FLAG: --healthz-port="10248"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866286    2583 flags.go:64] FLAG: --help="false"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866290    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866293    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866296    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866299    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866303    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 07 14:26:25.189676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866307    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866310    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866313    2583 flags.go:64] FLAG: --image-service-endpoint=""
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866316    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866319    2583 flags.go:64] FLAG: --kube-api-burst="100"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866322    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866325    2583 flags.go:64] FLAG: --kube-api-qps="50"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866328    2583 flags.go:64] FLAG: --kube-reserved=""
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866332    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866335    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866338    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866341    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866344    2583 flags.go:64] FLAG: --lock-file=""
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866346    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866349    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866352    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866358    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866361    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866364    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866367    2583 flags.go:64] FLAG: --logging-format="text"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866370    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866373    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866376    2583 flags.go:64] FLAG: --manifest-url=""
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866379    2583 flags.go:64] FLAG: --manifest-url-header=""
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866387    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 07 14:26:25.190417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866390    2583 flags.go:64] FLAG: --max-open-files="1000000"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866394    2583 flags.go:64] FLAG: --max-pods="110"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866398    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866401    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866403    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866407    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866410    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866412    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866415    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866422    2583 flags.go:64] FLAG: --node-status-max-images="50"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866425    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866428    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866431    2583 flags.go:64] FLAG: --pod-cidr=""
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866433    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866438    2583 flags.go:64] FLAG: --pod-manifest-path=""
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866441    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866444    2583 flags.go:64] FLAG: --pods-per-core="0"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866447    2583 flags.go:64] FLAG: --port="10250"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866450    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866453    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00294efc1520c3be2"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866456    2583 flags.go:64] FLAG: --qos-reserved=""
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866459    2583 flags.go:64] FLAG: --read-only-port="10255"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866462    2583 flags.go:64] FLAG: --register-node="true"
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866465    2583 flags.go:64] FLAG: --register-with-taints=""
Jul 07 14:26:25.191064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866468    2583 flags.go:64] FLAG: --registry-burst="10"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866472    2583 flags.go:64] FLAG: --registry-qps="5"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866475    2583 flags.go:64] FLAG: --reserved-cpus=""
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866478    2583 flags.go:64] FLAG: --reserved-memory=""
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866481    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866484    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866487    2583 flags.go:64] FLAG: --rotate-certificates="false"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866491    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866495    2583 flags.go:64] FLAG: --runonce="false"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866497    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866501    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866503    2583 flags.go:64] FLAG: --seccomp-default="false"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866506    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866509    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866512    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866515    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866517    2583 flags.go:64] FLAG: --storage-driver-password="root"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866520    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866523    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866526    2583 flags.go:64] FLAG: --storage-driver-user="root"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866528    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866531    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866534    2583 flags.go:64] FLAG: --system-cgroups=""
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866537    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866544    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 07 14:26:25.191751 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866547    2583 flags.go:64] FLAG: --tls-cert-file=""
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866549    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866554    2583 flags.go:64] FLAG: --tls-min-version=""
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866557    2583 flags.go:64] FLAG: --tls-private-key-file=""
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866559    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866562    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866565    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866568    2583 flags.go:64] FLAG: --v="2"
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866572    2583 flags.go:64] FLAG: --version="false"
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866576    2583 flags.go:64] FLAG: --vmodule=""
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866580    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866583    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866708    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866712    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866716    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866719    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866722    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866725    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866728    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866731    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866733    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866736    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:26:25.194864 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866738    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866741    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866745    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866748    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866751    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866754    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866757    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866759    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866762    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866766    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866768    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866771    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866773    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866776    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866778    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866781    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866784    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866786    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866789    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:26:25.195904 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866792    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866795    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866797    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866800    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866802    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866806    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866809    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866811    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866814    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866817    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866819    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866822    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866824    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866827    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866830    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866832    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866835    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866837    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866840    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866842    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:26:25.281516 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866845    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866847    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866851    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866854    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866856    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866859    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866861    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866864    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866866    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866869    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866873    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866876    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866879    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866882    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866885    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866888    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866891    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866894    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866897    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:26:25.282572 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866899    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866902    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866904    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866907    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866910    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866914    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866916    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866919    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866921    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866924    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866926    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866929    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866931    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866934    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866936    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866941    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866943    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866946    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866948    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866951    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:26:25.283201 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866953    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:26:25.283842 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866956    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:26:25.283842 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866958    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:26:25.283842 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866961    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:26:25.283842 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866963    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:26:25.283842 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866966    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:26:25.283842 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866969    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:26:25.283842 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866971    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:26:25.283842 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866975    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:26:25.283842 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866977    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:26:25.283842 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.866981    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:26:25.283842 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.866987    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 14:26:25.283842 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.874244    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 07 14:26:25.283842 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.874265    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 07 14:26:25.283842 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.874300    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.874309    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874369    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874375    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874378    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874381    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874385    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874387    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874390    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874393    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874396    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874398    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874402    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874406    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874410    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874413    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874416    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874419    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:26:25.284269 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874422    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874425    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874428    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874431    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874434    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874437    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874441    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874445    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874448    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874451    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874454    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874457    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874460    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874464    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874467    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874469    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874472    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874475    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874477    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:26:25.284901 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874480    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874483    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874485    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874488    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874491    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874493    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874496    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874498    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874501    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874503    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874506    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874509    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874511    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874514    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874516    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874519    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874521    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874528    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874531    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874534    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:26:25.285419 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874537    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874542    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874544    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874547    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874550    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874552    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874555    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874558    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874561    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874564    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874567    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874569    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874572    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874574    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874577    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874580    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874582    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874585    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874588    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874590    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:26:25.285974 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874593    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874595    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874598    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874601    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874603    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874606    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874609    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874611    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874614    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874619    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874622    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874625    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874627    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874630    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874632    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874635    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874637    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874640    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874643    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874645    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:26:25.286499 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874649    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:26:25.287053 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874651    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:26:25.287053 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874654    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:26:25.287053 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874656    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:26:25.287053 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.874662    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 14:26:25.287053 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874774    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 07 14:26:25.287053 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874778    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 07 14:26:25.287053 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874781    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 07 14:26:25.287053 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874784    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 07 14:26:25.287053 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874787    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 07 14:26:25.287053 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874790    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 07 14:26:25.287053 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874793    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 07 14:26:25.287053 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874795    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 07 14:26:25.287053 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874798    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 07 14:26:25.287053 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874801    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 07 14:26:25.287053 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874804    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874807    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874809    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874812    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874815    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874817    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874821    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874824    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874827    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874829    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874832    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874834    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874837    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874841    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874844    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874847    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874850    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874853    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874856    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 07 14:26:25.287510 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874859    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874863    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874866    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874869    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874872    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874875    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874877    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874880    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874882    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874885    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874888    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874890    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874893    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874896    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874898    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874901    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874903    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874906    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874908    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874912    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 07 14:26:25.288062 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874916    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874918    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874921    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874924    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874926    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874929    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874931    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874934    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874936    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874939    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874941    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874944    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874947    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874950    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874953    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874956    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874958    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874961    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874963    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874966    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 07 14:26:25.288602 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874968    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874971    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874973    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874976    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874978    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874981    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874983    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874986    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874988    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874991    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874994    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.874999    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875003    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875007    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875010    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875013    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875016    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875018    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875021    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 07 14:26:25.289187 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875023    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 07 14:26:25.289708 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875026    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 07 14:26:25.289708 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875028    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 07 14:26:25.289708 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875031    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 07 14:26:25.289708 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875033    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 07 14:26:25.289708 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875036    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 07 14:26:25.289708 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875039    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 07 14:26:25.289708 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875041    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 07 14:26:25.289708 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875044    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 07 14:26:25.289708 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875046    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 07 14:26:25.289708 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:24.875049    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 07 14:26:25.289708 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.875054    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 07 14:26:25.289708 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.875185    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 07 14:26:25.289708 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.877631    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 07 14:26:25.289708 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.879375    2583 server.go:1019] "Starting client certificate rotation"
Jul 07 14:26:25.290129 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.879479    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 14:26:25.290129 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.880393    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 07 14:26:25.290129 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.882166    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 14:26:25.290129 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.888737    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 07 14:26:25.290129 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.899466    2583 log.go:25] "Validated CRI v1 runtime API"
Jul 07 14:26:25.290129 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.901994    2583 log.go:25] "Validated CRI v1 image API"
Jul 07 14:26:25.290129 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.905625    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 07 14:26:25.290129 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.909081    2583 fs.go:135] Filesystem UUIDs: map[779a9f8f-9eff-4112-b42d-e1aa1e93d73b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9a5d899a-7746-423e-acdb-bbebd0305ac4:/dev/nvme0n1p4]
Jul 07 14:26:25.290129 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.909099    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 07 14:26:25.290129 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.915216    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.915756    2583 manager.go:216] Machine: {Timestamp:2026-07-07 14:26:24.914104083 +0000 UTC m=+0.377705280 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100142 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2296592056b77533e206413cda44c1 SystemUUID:ec229659-2056-b775-33e2-06413cda44c1 BootID:0bbdc632-e92c-441c-8c91-68a8b82a029c Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:98:7f:54:2a:cf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:98:7f:54:2a:cf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:4a:bc:19:d1:ec Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.915904    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.916028    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.919747    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.919772    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-1.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.920668    2583 topology_manager.go:138] "Creating topology manager with none policy"
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.920678    2583 container_manager_linux.go:309] "Creating device plugin manager"
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.920685    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.921548    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.923494    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.928328    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.928453    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.928485    2583 kubelet.go:486] "Attempting to sync node with API server"
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.928502    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.928517    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.928527    2583 kubelet.go:398] "Adding apiserver pod source"
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.928536    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 07 14:26:25.290551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.929024    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.929043    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.931640    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.934353    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.934391    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.935786    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.935798    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.935804    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.935810    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.935819    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.935825    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.935833    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.935838    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.935845    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.935850    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.935864    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.935870    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.936910    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.937102    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.940741    2583 server.go:1268] "Started kubelet"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.940825    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.941090    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.941138    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.943187    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.943230    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-1.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:24.943337    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-1.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:24.943374    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.943601    2583 server.go:310] "Adding debug handlers to kubelet server"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.947047    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9lfqm"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:24.947991    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:24.949366    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-1.ec2.internal.18c0081fc29b6d96  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-1.ec2.internal,UID:ip-10-0-129-1.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-1.ec2.internal,},FirstTimestamp:2026-07-07 14:26:24.940690838 +0000 UTC m=+0.404292035,LastTimestamp:2026-07-07 14:26:24.940690838 +0000 UTC m=+0.404292035,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-1.ec2.internal,}"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.952154    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.952155    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:24.952235    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-1.ec2.internal\" not found"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.952983    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.953020    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.953088    2583 reconstruct.go:97] "Volume reconstruction finished"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.953096    2583 reconciler.go:29] "Reconciler: start to sync state"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.953134    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.954759    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9lfqm"
Jul 07 14:26:25.291066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.955132    2583 factory.go:55] Registering systemd factory
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.955144    2583 factory.go:223] Registration of the systemd container factory successfully
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:24.955504    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.955949    2583 factory.go:153] Registering CRI-O factory
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.955966    2583 factory.go:223] Registration of the crio container factory successfully
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.956031    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.956063    2583 factory.go:103] Registering Raw factory
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.956078    2583 manager.go:1196] Started watching for new ooms in manager
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.956483    2583 manager.go:349] Starting recovery of all containers
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.967738    2583 manager.go:354] Recovery completed
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:24.969468    2583 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-1.ec2.internal\" not found" node="ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.976327    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.978100    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.978129    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.978139    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.978702    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.978711    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.978727    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.978703    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.979460    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.979480    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.979489    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.980744    2583 policy_none.go:49] "None policy: Start"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.980756    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.980765    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:24.982738    2583 policy_none.go:47] "Start"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.010431    2583 manager.go:337] "Starting Device Plugin manager"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:25.010459    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.010469    2583 server.go:87] "Starting device plugin registration server"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.010749    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.010761    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.010892    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.010994    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.011009    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:25.011591    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:25.011631    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-1.ec2.internal\" not found"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.092439    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.093574    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.093609    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.093633    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:25.093669    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.096250    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.111452    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.112416    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.112449    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.112461    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.112490    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.292148 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.124888    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:25.124914    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-1.ec2.internal\": node \"ip-10-0-129-1.ec2.internal\" not found"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:25.148557    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-1.ec2.internal\" not found"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.194522    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-1.ec2.internal"]
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.194861    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.195992    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.196027    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.196037    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.198372    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.198569    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.198601    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.199417    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.199439    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.199448    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.199782    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.199801    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.199812    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.201692    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.201716    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.202484    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasSufficientMemory"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.202504    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasNoDiskPressure"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.202516    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeHasSufficientPID"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:25.242473    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-1.ec2.internal\" not found" node="ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:25.248643    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-1.ec2.internal\" not found" node="ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:25.251480    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-1.ec2.internal\" not found"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.254741    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e20e813e5c8ec59bb9d94b29eb79c2ad-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal\" (UID: \"e20e813e5c8ec59bb9d94b29eb79c2ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.254768    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/603217e51a894b3722465edfd41817b7-config\") pod \"kube-apiserver-proxy-ip-10-0-129-1.ec2.internal\" (UID: \"603217e51a894b3722465edfd41817b7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.293612 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.254785    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e20e813e5c8ec59bb9d94b29eb79c2ad-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal\" (UID: \"e20e813e5c8ec59bb9d94b29eb79c2ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.376944 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:25.352591    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-1.ec2.internal\" not found"
Jul 07 14:26:25.376944 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.355909    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e20e813e5c8ec59bb9d94b29eb79c2ad-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal\" (UID: \"e20e813e5c8ec59bb9d94b29eb79c2ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.376944 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.355935    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e20e813e5c8ec59bb9d94b29eb79c2ad-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal\" (UID: \"e20e813e5c8ec59bb9d94b29eb79c2ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.376944 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.355954    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/603217e51a894b3722465edfd41817b7-config\") pod \"kube-apiserver-proxy-ip-10-0-129-1.ec2.internal\" (UID: \"603217e51a894b3722465edfd41817b7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.376944 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.355994    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/603217e51a894b3722465edfd41817b7-config\") pod \"kube-apiserver-proxy-ip-10-0-129-1.ec2.internal\" (UID: \"603217e51a894b3722465edfd41817b7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.376944 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.356012    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e20e813e5c8ec59bb9d94b29eb79c2ad-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal\" (UID: \"e20e813e5c8ec59bb9d94b29eb79c2ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.376944 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.356022    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e20e813e5c8ec59bb9d94b29eb79c2ad-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal\" (UID: \"e20e813e5c8ec59bb9d94b29eb79c2ad\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.480436 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:25.453317    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-1.ec2.internal\" not found"
Jul 07 14:26:25.575599 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.545685    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.575599 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.551527    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:25.575599 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:25.555853    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-1.ec2.internal\" not found"
Jul 07 14:26:25.658218 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:25.656801    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-1.ec2.internal\" not found"
Jul 07 14:26:26.821341 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:25.757313    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-1.ec2.internal\" not found"
Jul 07 14:26:26.821341 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:25.782024    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:26:26.894550 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.824420    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:26:26.894550 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.853010    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:26.894550 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.869592    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 07 14:26:26.894550 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.871157    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:26.894550 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.879368    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 07 14:26:26.894550 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.879478    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 14:26:26.894550 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.879504    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 14:26:26.894550 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.879530    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 07 14:26:26.894550 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:26.879579    2583 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a6ef97384509d4ad6aff53915b6cedbb-3fb2c87ed1a48548.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-machine-config-operator/pods\": read tcp 10.0.129.1:42058->23.23.62.122:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.929380    2583 apiserver.go:52] "Watching apiserver"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.945145    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.945856    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-mk6nm","kube-system/konnectivity-agent-sr65q","kube-system/kube-apiserver-proxy-ip-10-0-129-1.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287","openshift-dns/node-resolver-m2nzp","openshift-multus/multus-9p6fd","openshift-network-diagnostics/network-check-target-gp4zb","openshift-ovn-kubernetes/ovnkube-node-h577t","openshift-cluster-node-tuning-operator/tuned-8blpl","openshift-image-registry/node-ca-28k4c","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal","openshift-multus/multus-additional-cni-plugins-k5mt8","openshift-multus/network-metrics-daemon-8sq9p","openshift-network-operator/iptables-alerter-g62rg"]
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.948507    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:26.948571    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.950564    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sr65q"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.952108    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.952137    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9gqcj\""
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.952573    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.952643    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.952951    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.954383    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.954814    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.954970    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m2nzp"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.955439    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.955553    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vfm84\""
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.956881    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.957539    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.957664    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.957790    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sdw92\""
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.958308    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.958559    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 14:21:25 +0000 UTC" deadline="2028-02-14 18:33:58.746267502 +0000 UTC"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.958590    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14092h7m31.787679698s"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.958824    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.959030    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:26.959069    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.959345    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.959572    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.959684    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5q99j\""
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.962335    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.963354    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:26.994486 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.965217    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.965321    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.965409    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.965553    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-28k4c"
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.965688    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.965803    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.965994    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xx5hm\""
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.965996    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.966061    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.966021    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t4z46\""
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.966877    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.967849    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.969863    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.969885    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-etc-selinux\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.969902    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/324585d5-c3f9-4d3a-97b6-01755df3f0bd-hosts-file\") pod \"node-resolver-m2nzp\" (UID: \"324585d5-c3f9-4d3a-97b6-01755df3f0bd\") " pod="openshift-dns/node-resolver-m2nzp"
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.969924    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e360a2f-655d-464f-98e3-00096c75fbe9-dbus\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.969947    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06eab125-4856-449a-982f-111e9231f1d9-agent-certs\") pod \"konnectivity-agent-sr65q\" (UID: \"06eab125-4856-449a-982f-111e9231f1d9\") " pod="kube-system/konnectivity-agent-sr65q"
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.969993    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06eab125-4856-449a-982f-111e9231f1d9-konnectivity-ca\") pod \"konnectivity-agent-sr65q\" (UID: \"06eab125-4856-449a-982f-111e9231f1d9\") " pod="kube-system/konnectivity-agent-sr65q"
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.970017    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-registration-dir\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.970039    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.970085    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:26.970094    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.970039    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-sys-fs\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:26.995686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.970141    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp45q\" (UniqueName: \"kubernetes.io/projected/324585d5-c3f9-4d3a-97b6-01755df3f0bd-kube-api-access-hp45q\") pod \"node-resolver-m2nzp\" (UID: \"324585d5-c3f9-4d3a-97b6-01755df3f0bd\") " pod="openshift-dns/node-resolver-m2nzp"
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.970170    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e360a2f-655d-464f-98e3-00096c75fbe9-kubelet-config\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.970195    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-device-dir\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.970221    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc7qs\" (UniqueName: \"kubernetes.io/projected/14321cfa-5c8b-408a-a66c-eb1c37d702e6-kube-api-access-cc7qs\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.970244    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/324585d5-c3f9-4d3a-97b6-01755df3f0bd-tmp-dir\") pod \"node-resolver-m2nzp\" (UID: \"324585d5-c3f9-4d3a-97b6-01755df3f0bd\") " pod="openshift-dns/node-resolver-m2nzp"
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.970268    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.970290    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-socket-dir\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.970351    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.970379    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.970544    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.970598    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gfjqt\""
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.981090    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4t5rd\""
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.981114    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g62rg"
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.981490    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.983591    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.983676    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.984956    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c4tjf\""
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.984970    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:26:26.996858 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:26.985075    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.001997    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mj59p"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.012884    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mj59p"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.055383    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070531    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-system-cni-dir\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070554    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-multus-cni-dir\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070573    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-multus-socket-dir-parent\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070596    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf8gp\" (UniqueName: \"kubernetes.io/projected/92c6df4a-e05d-4526-97da-de6c79fbe402-kube-api-access-qf8gp\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070619    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-systemd-units\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070637    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-sysconfig\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070650    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-kubernetes\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070669    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06eab125-4856-449a-982f-111e9231f1d9-agent-certs\") pod \"konnectivity-agent-sr65q\" (UID: \"06eab125-4856-449a-982f-111e9231f1d9\") " pod="kube-system/konnectivity-agent-sr65q"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070691    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-multus-conf-dir\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070708    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkqq2\" (UniqueName: \"kubernetes.io/projected/3d0c2bd8-2635-4560-9823-91baebc88fbf-kube-api-access-rkqq2\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070730    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/13805be4-13ec-42d1-a93d-b7fff2258a24-serviceca\") pod \"node-ca-28k4c\" (UID: \"13805be4-13ec-42d1-a93d-b7fff2258a24\") " pod="openshift-image-registry/node-ca-28k4c"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070754    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-var-lib-openvswitch\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070775    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-run-openvswitch\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070789    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-cni-bin\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070809    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-sysctl-d\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.083269 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070831    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-run-k8s-cni-cncf-io\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070871    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c53d55ac-10e7-40fa-a2b5-e027292cda95-cni-binary-copy\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070897    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-slash\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070928    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-run-ovn-kubernetes\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070945    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-cni-netd\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070964    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.070998    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-etc-selinux\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071016    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-hostroot\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071032    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-run-multus-certs\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071049    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-log-socket\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.071065    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071067    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c39143b3-6865-46e6-971c-9c33afb34b45-env-overrides\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071141    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-etc-selinux\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.071146    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret podName:7e360a2f-655d-464f-98e3-00096c75fbe9 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:27.571115344 +0000 UTC m=+2.034015267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret") pod "global-pull-secret-syncer-mk6nm" (UID: "7e360a2f-655d-464f-98e3-00096c75fbe9") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071184    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c39143b3-6865-46e6-971c-9c33afb34b45-ovn-node-metrics-cert\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071201    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-sys\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.084181 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071222    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06eab125-4856-449a-982f-111e9231f1d9-konnectivity-ca\") pod \"konnectivity-agent-sr65q\" (UID: \"06eab125-4856-449a-982f-111e9231f1d9\") " pod="kube-system/konnectivity-agent-sr65q"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071245    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp45q\" (UniqueName: \"kubernetes.io/projected/324585d5-c3f9-4d3a-97b6-01755df3f0bd-kube-api-access-hp45q\") pod \"node-resolver-m2nzp\" (UID: \"324585d5-c3f9-4d3a-97b6-01755df3f0bd\") " pod="openshift-dns/node-resolver-m2nzp"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071261    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c53d55ac-10e7-40fa-a2b5-e027292cda95-os-release\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071281    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-var-lib-kubelet\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071299    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-tuned\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071334    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc7v9\" (UniqueName: \"kubernetes.io/projected/3b53ae89-367b-4c2a-99f2-6bf60db0f958-kube-api-access-jc7v9\") pod \"network-metrics-daemon-8sq9p\" (UID: \"3b53ae89-367b-4c2a-99f2-6bf60db0f958\") " pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071365    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtn9t\" (UniqueName: \"kubernetes.io/projected/e0def9a5-886b-490c-8a02-32e9f5dcecc5-kube-api-access-dtn9t\") pod \"iptables-alerter-g62rg\" (UID: \"e0def9a5-886b-490c-8a02-32e9f5dcecc5\") " pod="openshift-network-operator/iptables-alerter-g62rg"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071394    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e360a2f-655d-464f-98e3-00096c75fbe9-kubelet-config\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071431    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-var-lib-cni-multus\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071446    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13805be4-13ec-42d1-a93d-b7fff2258a24-host\") pod \"node-ca-28k4c\" (UID: \"13805be4-13ec-42d1-a93d-b7fff2258a24\") " pod="openshift-image-registry/node-ca-28k4c"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071458    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d0c2bd8-2635-4560-9823-91baebc88fbf-tmp\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071486    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0def9a5-886b-490c-8a02-32e9f5dcecc5-host-slash\") pod \"iptables-alerter-g62rg\" (UID: \"e0def9a5-886b-490c-8a02-32e9f5dcecc5\") " pod="openshift-network-operator/iptables-alerter-g62rg"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071513    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071519    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e360a2f-655d-464f-98e3-00096c75fbe9-kubelet-config\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071530    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-cnibin\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071564    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-os-release\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071598    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:27.086659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071611    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-run-netns\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071638    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-kubelet\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071660    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-run-systemd\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071674    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-node-log\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071700    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071731    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-registration-dir\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071750    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-sys-fs\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071768    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-var-lib-cni-bin\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071783    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c53d55ac-10e7-40fa-a2b5-e027292cda95-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071787    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071809    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c53d55ac-10e7-40fa-a2b5-e027292cda95-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071834    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c39143b3-6865-46e6-971c-9c33afb34b45-ovnkube-config\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071841    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-sys-fs\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071856    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-sysctl-conf\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071786    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-registration-dir\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071882    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-systemd\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071917    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-device-dir\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:27.087499 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071931    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-device-dir\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071940    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/92c6df4a-e05d-4526-97da-de6c79fbe402-multus-daemon-config\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071963    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c53d55ac-10e7-40fa-a2b5-e027292cda95-system-cni-dir\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.071992    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c53d55ac-10e7-40fa-a2b5-e027292cda95-cnibin\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072011    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t27f\" (UniqueName: \"kubernetes.io/projected/c53d55ac-10e7-40fa-a2b5-e027292cda95-kube-api-access-5t27f\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072033    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-etc-openvswitch\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072046    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-run\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072059    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-lib-modules\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072090    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-run-ovn\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072096    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/06eab125-4856-449a-982f-111e9231f1d9-konnectivity-ca\") pod \"konnectivity-agent-sr65q\" (UID: \"06eab125-4856-449a-982f-111e9231f1d9\") " pod="kube-system/konnectivity-agent-sr65q"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072105    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-host\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072120    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-modprobe-d\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072140    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e360a2f-655d-464f-98e3-00096c75fbe9-dbus\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072159    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-etc-kubernetes\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072174    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zz57\" (UniqueName: \"kubernetes.io/projected/13805be4-13ec-42d1-a93d-b7fff2258a24-kube-api-access-7zz57\") pod \"node-ca-28k4c\" (UID: \"13805be4-13ec-42d1-a93d-b7fff2258a24\") " pod="openshift-image-registry/node-ca-28k4c"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072188    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c53d55ac-10e7-40fa-a2b5-e027292cda95-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.089466 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072203    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwvdh\" (UniqueName: \"kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh\") pod \"network-check-target-gp4zb\" (UID: \"1c687b00-93cd-4401-a4c7-04aa5c966647\") " pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072224    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs\") pod \"network-metrics-daemon-8sq9p\" (UID: \"3b53ae89-367b-4c2a-99f2-6bf60db0f958\") " pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072241    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc7qs\" (UniqueName: \"kubernetes.io/projected/14321cfa-5c8b-408a-a66c-eb1c37d702e6-kube-api-access-cc7qs\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072254    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/324585d5-c3f9-4d3a-97b6-01755df3f0bd-tmp-dir\") pod \"node-resolver-m2nzp\" (UID: \"324585d5-c3f9-4d3a-97b6-01755df3f0bd\") " pod="openshift-dns/node-resolver-m2nzp"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072271    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c39143b3-6865-46e6-971c-9c33afb34b45-ovnkube-script-lib\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072297    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g66xl\" (UniqueName: \"kubernetes.io/projected/c39143b3-6865-46e6-971c-9c33afb34b45-kube-api-access-g66xl\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072322    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-socket-dir\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072332    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e360a2f-655d-464f-98e3-00096c75fbe9-dbus\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072345    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92c6df4a-e05d-4526-97da-de6c79fbe402-cni-binary-copy\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072367    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-var-lib-kubelet\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072392    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-run-netns\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072417    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e0def9a5-886b-490c-8a02-32e9f5dcecc5-iptables-alerter-script\") pod \"iptables-alerter-g62rg\" (UID: \"e0def9a5-886b-490c-8a02-32e9f5dcecc5\") " pod="openshift-network-operator/iptables-alerter-g62rg"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072442    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/324585d5-c3f9-4d3a-97b6-01755df3f0bd-hosts-file\") pod \"node-resolver-m2nzp\" (UID: \"324585d5-c3f9-4d3a-97b6-01755df3f0bd\") " pod="openshift-dns/node-resolver-m2nzp"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072500    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/14321cfa-5c8b-408a-a66c-eb1c37d702e6-socket-dir\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072530    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/324585d5-c3f9-4d3a-97b6-01755df3f0bd-tmp-dir\") pod \"node-resolver-m2nzp\" (UID: \"324585d5-c3f9-4d3a-97b6-01755df3f0bd\") " pod="openshift-dns/node-resolver-m2nzp"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.072538    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/324585d5-c3f9-4d3a-97b6-01755df3f0bd-hosts-file\") pod \"node-resolver-m2nzp\" (UID: \"324585d5-c3f9-4d3a-97b6-01755df3f0bd\") " pod="openshift-dns/node-resolver-m2nzp"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.075625    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/06eab125-4856-449a-982f-111e9231f1d9-agent-certs\") pod \"konnectivity-agent-sr65q\" (UID: \"06eab125-4856-449a-982f-111e9231f1d9\") " pod="kube-system/konnectivity-agent-sr65q"
Jul 07 14:26:27.089956 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.078716    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp45q\" (UniqueName: \"kubernetes.io/projected/324585d5-c3f9-4d3a-97b6-01755df3f0bd-kube-api-access-hp45q\") pod \"node-resolver-m2nzp\" (UID: \"324585d5-c3f9-4d3a-97b6-01755df3f0bd\") " pod="openshift-dns/node-resolver-m2nzp"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.079132    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc7qs\" (UniqueName: \"kubernetes.io/projected/14321cfa-5c8b-408a-a66c-eb1c37d702e6-kube-api-access-cc7qs\") pod \"aws-ebs-csi-driver-node-dn287\" (UID: \"14321cfa-5c8b-408a-a66c-eb1c37d702e6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.172737    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-node-log\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.172772    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.172792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-var-lib-cni-bin\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.172853    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.172916    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-var-lib-cni-bin\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.172963    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c53d55ac-10e7-40fa-a2b5-e027292cda95-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173006    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-node-log\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173016    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c53d55ac-10e7-40fa-a2b5-e027292cda95-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173051    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c39143b3-6865-46e6-971c-9c33afb34b45-ovnkube-config\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173075    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-sysctl-conf\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173096    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-systemd\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173114    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c53d55ac-10e7-40fa-a2b5-e027292cda95-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173119    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/92c6df4a-e05d-4526-97da-de6c79fbe402-multus-daemon-config\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173162    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c53d55ac-10e7-40fa-a2b5-e027292cda95-system-cni-dir\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173186    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c53d55ac-10e7-40fa-a2b5-e027292cda95-cnibin\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.186290 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173209    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t27f\" (UniqueName: \"kubernetes.io/projected/c53d55ac-10e7-40fa-a2b5-e027292cda95-kube-api-access-5t27f\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173234    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-etc-openvswitch\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173237    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-sysctl-conf\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173255    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-run\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173279    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-lib-modules\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173299    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-run-ovn\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173318    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-host\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173341    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-modprobe-d\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173363    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-etc-kubernetes\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173384    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zz57\" (UniqueName: \"kubernetes.io/projected/13805be4-13ec-42d1-a93d-b7fff2258a24-kube-api-access-7zz57\") pod \"node-ca-28k4c\" (UID: \"13805be4-13ec-42d1-a93d-b7fff2258a24\") " pod="openshift-image-registry/node-ca-28k4c"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173404    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c53d55ac-10e7-40fa-a2b5-e027292cda95-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173427    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwvdh\" (UniqueName: \"kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh\") pod \"network-check-target-gp4zb\" (UID: \"1c687b00-93cd-4401-a4c7-04aa5c966647\") " pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173449    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs\") pod \"network-metrics-daemon-8sq9p\" (UID: \"3b53ae89-367b-4c2a-99f2-6bf60db0f958\") " pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173473    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c39143b3-6865-46e6-971c-9c33afb34b45-ovnkube-script-lib\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173493    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g66xl\" (UniqueName: \"kubernetes.io/projected/c39143b3-6865-46e6-971c-9c33afb34b45-kube-api-access-g66xl\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173515    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92c6df4a-e05d-4526-97da-de6c79fbe402-cni-binary-copy\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173535    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-var-lib-kubelet\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.187587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173563    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-run-netns\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173584    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e0def9a5-886b-490c-8a02-32e9f5dcecc5-iptables-alerter-script\") pod \"iptables-alerter-g62rg\" (UID: \"e0def9a5-886b-490c-8a02-32e9f5dcecc5\") " pod="openshift-network-operator/iptables-alerter-g62rg"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173605    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-system-cni-dir\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173605    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c39143b3-6865-46e6-971c-9c33afb34b45-ovnkube-config\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173612    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c53d55ac-10e7-40fa-a2b5-e027292cda95-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173624    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-multus-cni-dir\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173671    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-systemd\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173698    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-multus-socket-dir-parent\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173723    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf8gp\" (UniqueName: \"kubernetes.io/projected/92c6df4a-e05d-4526-97da-de6c79fbe402-kube-api-access-qf8gp\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173745    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-systemd-units\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173768    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-sysconfig\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173790    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-kubernetes\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173815    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-multus-conf-dir\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173841    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkqq2\" (UniqueName: \"kubernetes.io/projected/3d0c2bd8-2635-4560-9823-91baebc88fbf-kube-api-access-rkqq2\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173865    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/13805be4-13ec-42d1-a93d-b7fff2258a24-serviceca\") pod \"node-ca-28k4c\" (UID: \"13805be4-13ec-42d1-a93d-b7fff2258a24\") " pod="openshift-image-registry/node-ca-28k4c"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173889    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-var-lib-openvswitch\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173899    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c53d55ac-10e7-40fa-a2b5-e027292cda95-system-cni-dir\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.188254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173913    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-run-openvswitch\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173936    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-cni-bin\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173943    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c53d55ac-10e7-40fa-a2b5-e027292cda95-cnibin\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.173964    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-sysctl-d\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174008    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-run-k8s-cni-cncf-io\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174033    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c53d55ac-10e7-40fa-a2b5-e027292cda95-cni-binary-copy\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174055    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-slash\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174079    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c53d55ac-10e7-40fa-a2b5-e027292cda95-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174100    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-run-ovn-kubernetes\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174138    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-cni-netd\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174148    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-sysconfig\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174174    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-etc-openvswitch\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174189    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-cni-netd\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174194    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-kubernetes\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174216    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-run\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174105    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-systemd-units\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174323    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-run-netns\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174370    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-var-lib-kubelet\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.189725 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174396    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-etc-kubernetes\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174430    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-run-ovn-kubernetes\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174455    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/92c6df4a-e05d-4526-97da-de6c79fbe402-multus-daemon-config\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174474    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-multus-socket-dir-parent\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174508    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-cni-bin\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174510    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-multus-conf-dir\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174544    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-run-openvswitch\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174578    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-slash\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.174582    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174607    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-lib-modules\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174611    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-run-k8s-cni-cncf-io\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174639    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-run-ovn\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.174685    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs podName:3b53ae89-367b-4c2a-99f2-6bf60db0f958 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:27.674666663 +0000 UTC m=+2.137566598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs") pod "network-metrics-daemon-8sq9p" (UID: "3b53ae89-367b-4c2a-99f2-6bf60db0f958") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174740    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-sysctl-d\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174763    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e0def9a5-886b-490c-8a02-32e9f5dcecc5-iptables-alerter-script\") pod \"iptables-alerter-g62rg\" (UID: \"e0def9a5-886b-490c-8a02-32e9f5dcecc5\") " pod="openshift-network-operator/iptables-alerter-g62rg"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174783    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-system-cni-dir\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174806    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92c6df4a-e05d-4526-97da-de6c79fbe402-cni-binary-copy\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174819    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-host\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.190367 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174824    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-hostroot\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174852    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-run-multus-certs\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174900    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-multus-cni-dir\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174927    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-var-lib-openvswitch\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.174935    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-log-socket\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175015    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c39143b3-6865-46e6-971c-9c33afb34b45-env-overrides\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175025    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-modprobe-d\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175035    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-run-multus-certs\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175039    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c39143b3-6865-46e6-971c-9c33afb34b45-ovn-node-metrics-cert\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175063    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-log-socket\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175094    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-sys\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175133    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c53d55ac-10e7-40fa-a2b5-e027292cda95-os-release\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175159    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-var-lib-kubelet\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175160    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/13805be4-13ec-42d1-a93d-b7fff2258a24-serviceca\") pod \"node-ca-28k4c\" (UID: \"13805be4-13ec-42d1-a93d-b7fff2258a24\") " pod="openshift-image-registry/node-ca-28k4c"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175180    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c53d55ac-10e7-40fa-a2b5-e027292cda95-cni-binary-copy\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175182    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-tuned\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175284    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c53d55ac-10e7-40fa-a2b5-e027292cda95-os-release\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.191429 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175330    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc7v9\" (UniqueName: \"kubernetes.io/projected/3b53ae89-367b-4c2a-99f2-6bf60db0f958-kube-api-access-jc7v9\") pod \"network-metrics-daemon-8sq9p\" (UID: \"3b53ae89-367b-4c2a-99f2-6bf60db0f958\") " pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175362    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtn9t\" (UniqueName: \"kubernetes.io/projected/e0def9a5-886b-490c-8a02-32e9f5dcecc5-kube-api-access-dtn9t\") pod \"iptables-alerter-g62rg\" (UID: \"e0def9a5-886b-490c-8a02-32e9f5dcecc5\") " pod="openshift-network-operator/iptables-alerter-g62rg"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175379    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c39143b3-6865-46e6-971c-9c33afb34b45-ovnkube-script-lib\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175391    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-var-lib-cni-multus\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175424    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13805be4-13ec-42d1-a93d-b7fff2258a24-host\") pod \"node-ca-28k4c\" (UID: \"13805be4-13ec-42d1-a93d-b7fff2258a24\") " pod="openshift-image-registry/node-ca-28k4c"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175448    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d0c2bd8-2635-4560-9823-91baebc88fbf-tmp\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175471    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0def9a5-886b-490c-8a02-32e9f5dcecc5-host-slash\") pod \"iptables-alerter-g62rg\" (UID: \"e0def9a5-886b-490c-8a02-32e9f5dcecc5\") " pod="openshift-network-operator/iptables-alerter-g62rg"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175496    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-cnibin\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175505    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-hostroot\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175567    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0def9a5-886b-490c-8a02-32e9f5dcecc5-host-slash\") pod \"iptables-alerter-g62rg\" (UID: \"e0def9a5-886b-490c-8a02-32e9f5dcecc5\") " pod="openshift-network-operator/iptables-alerter-g62rg"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175554    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-os-release\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175608    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-run-netns\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175612    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c39143b3-6865-46e6-971c-9c33afb34b45-env-overrides\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175637    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-kubelet\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175618    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-cnibin\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175674    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-run-systemd\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175679    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-run-netns\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175714    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-host-kubelet\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.192301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175676    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-host-var-lib-cni-multus\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.193371 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175766    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-sys\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.193371 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175771    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13805be4-13ec-42d1-a93d-b7fff2258a24-host\") pod \"node-ca-28k4c\" (UID: \"13805be4-13ec-42d1-a93d-b7fff2258a24\") " pod="openshift-image-registry/node-ca-28k4c"
Jul 07 14:26:27.193371 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175797    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d0c2bd8-2635-4560-9823-91baebc88fbf-var-lib-kubelet\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.193371 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175818    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c39143b3-6865-46e6-971c-9c33afb34b45-run-systemd\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.193371 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.175851    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92c6df4a-e05d-4526-97da-de6c79fbe402-os-release\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.193371 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.177671    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3d0c2bd8-2635-4560-9823-91baebc88fbf-tmp\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.193371 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.177721    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3d0c2bd8-2635-4560-9823-91baebc88fbf-etc-tuned\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.193371 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.180233    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c39143b3-6865-46e6-971c-9c33afb34b45-ovn-node-metrics-cert\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.193371 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.191873    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t27f\" (UniqueName: \"kubernetes.io/projected/c53d55ac-10e7-40fa-a2b5-e027292cda95-kube-api-access-5t27f\") pod \"multus-additional-cni-plugins-k5mt8\" (UID: \"c53d55ac-10e7-40fa-a2b5-e027292cda95\") " pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.194883 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.194859    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:26:27.194883 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.194881    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:26:27.195109 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.194894    2583 projected.go:196] Error preparing data for projected volume kube-api-access-pwvdh for pod openshift-network-diagnostics/network-check-target-gp4zb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:26:27.195109 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.194962    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh podName:1c687b00-93cd-4401-a4c7-04aa5c966647 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:27.694943178 +0000 UTC m=+2.157843117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pwvdh" (UniqueName: "kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh") pod "network-check-target-gp4zb" (UID: "1c687b00-93cd-4401-a4c7-04aa5c966647") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:26:27.195676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.195654    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkqq2\" (UniqueName: \"kubernetes.io/projected/3d0c2bd8-2635-4560-9823-91baebc88fbf-kube-api-access-rkqq2\") pod \"tuned-8blpl\" (UID: \"3d0c2bd8-2635-4560-9823-91baebc88fbf\") " pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.197268 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.197145    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc7v9\" (UniqueName: \"kubernetes.io/projected/3b53ae89-367b-4c2a-99f2-6bf60db0f958-kube-api-access-jc7v9\") pod \"network-metrics-daemon-8sq9p\" (UID: \"3b53ae89-367b-4c2a-99f2-6bf60db0f958\") " pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:27.197456 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.197437    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtn9t\" (UniqueName: \"kubernetes.io/projected/e0def9a5-886b-490c-8a02-32e9f5dcecc5-kube-api-access-dtn9t\") pod \"iptables-alerter-g62rg\" (UID: \"e0def9a5-886b-490c-8a02-32e9f5dcecc5\") " pod="openshift-network-operator/iptables-alerter-g62rg"
Jul 07 14:26:27.198351 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.198328    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zz57\" (UniqueName: \"kubernetes.io/projected/13805be4-13ec-42d1-a93d-b7fff2258a24-kube-api-access-7zz57\") pod \"node-ca-28k4c\" (UID: \"13805be4-13ec-42d1-a93d-b7fff2258a24\") " pod="openshift-image-registry/node-ca-28k4c"
Jul 07 14:26:27.198762 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.198741    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf8gp\" (UniqueName: \"kubernetes.io/projected/92c6df4a-e05d-4526-97da-de6c79fbe402-kube-api-access-qf8gp\") pod \"multus-9p6fd\" (UID: \"92c6df4a-e05d-4526-97da-de6c79fbe402\") " pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.199154 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.199139    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g66xl\" (UniqueName: \"kubernetes.io/projected/c39143b3-6865-46e6-971c-9c33afb34b45-kube-api-access-g66xl\") pod \"ovnkube-node-h577t\" (UID: \"c39143b3-6865-46e6-971c-9c33afb34b45\") " pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.249593 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.249567    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8blpl"
Jul 07 14:26:27.256232 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.256204    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-28k4c"
Jul 07 14:26:27.277798 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:27.277764    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20e813e5c8ec59bb9d94b29eb79c2ad.slice/crio-c7b14421cadb4be7ee447b77a6f4d5449d9c896ca0cf1b6f05f9b3b550fea665 WatchSource:0}: Error finding container c7b14421cadb4be7ee447b77a6f4d5449d9c896ca0cf1b6f05f9b3b550fea665: Status 404 returned error can't find the container with id c7b14421cadb4be7ee447b77a6f4d5449d9c896ca0cf1b6f05f9b3b550fea665
Jul 07 14:26:27.278421 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.278404    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k5mt8"
Jul 07 14:26:27.278518 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:27.278486    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d0c2bd8_2635_4560_9823_91baebc88fbf.slice/crio-f5361544ae9011730399d50005c74cc2e53d4a6cd1b2e36ca762f27f5a8fb9b2 WatchSource:0}: Error finding container f5361544ae9011730399d50005c74cc2e53d4a6cd1b2e36ca762f27f5a8fb9b2: Status 404 returned error can't find the container with id f5361544ae9011730399d50005c74cc2e53d4a6cd1b2e36ca762f27f5a8fb9b2
Jul 07 14:26:27.278711 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:27.278683    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13805be4_13ec_42d1_a93d_b7fff2258a24.slice/crio-4374a043cf22c20a779e60cf95750ba629e760216ee9d6052c3f6b97827309a4 WatchSource:0}: Error finding container 4374a043cf22c20a779e60cf95750ba629e760216ee9d6052c3f6b97827309a4: Status 404 returned error can't find the container with id 4374a043cf22c20a779e60cf95750ba629e760216ee9d6052c3f6b97827309a4
Jul 07 14:26:27.278931 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:27.278915    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod603217e51a894b3722465edfd41817b7.slice/crio-daa1054fd8e52950b7bc97d24cd2b146c510f87c15462bbf8ab1efab237fe8d0 WatchSource:0}: Error finding container daa1054fd8e52950b7bc97d24cd2b146c510f87c15462bbf8ab1efab237fe8d0: Status 404 returned error can't find the container with id daa1054fd8e52950b7bc97d24cd2b146c510f87c15462bbf8ab1efab237fe8d0
Jul 07 14:26:27.283122 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.283093    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 14:26:27.285040 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.285020    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g62rg"
Jul 07 14:26:27.290931 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:27.290906    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc53d55ac_10e7_40fa_a2b5_e027292cda95.slice/crio-06bcf3f2ca62fe15ebbb17b8861c74fd339627e265ba1293edf9cebb3ec62520 WatchSource:0}: Error finding container 06bcf3f2ca62fe15ebbb17b8861c74fd339627e265ba1293edf9cebb3ec62520: Status 404 returned error can't find the container with id 06bcf3f2ca62fe15ebbb17b8861c74fd339627e265ba1293edf9cebb3ec62520
Jul 07 14:26:27.295467 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:27.295449    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0def9a5_886b_490c_8a02_32e9f5dcecc5.slice/crio-b2c92af130e4b0260528f7ea935b768d964c662f67ead2958f3e592fe95f5c59 WatchSource:0}: Error finding container b2c92af130e4b0260528f7ea935b768d964c662f67ead2958f3e592fe95f5c59: Status 404 returned error can't find the container with id b2c92af130e4b0260528f7ea935b768d964c662f67ead2958f3e592fe95f5c59
Jul 07 14:26:27.317804 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.317775    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sr65q"
Jul 07 14:26:27.326785 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:27.326761    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06eab125_4856_449a_982f_111e9231f1d9.slice/crio-83a6d2a176aec8344b95e2051c7ceac96bf5466c12fa25d61e8336534e71976b WatchSource:0}: Error finding container 83a6d2a176aec8344b95e2051c7ceac96bf5466c12fa25d61e8336534e71976b: Status 404 returned error can't find the container with id 83a6d2a176aec8344b95e2051c7ceac96bf5466c12fa25d61e8336534e71976b
Jul 07 14:26:27.337844 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.337790    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287"
Jul 07 14:26:27.346154 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:27.346130    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14321cfa_5c8b_408a_a66c_eb1c37d702e6.slice/crio-0102bee251e9376d4c51952774db0b44b4576d32060647984c127d3d46cbd376 WatchSource:0}: Error finding container 0102bee251e9376d4c51952774db0b44b4576d32060647984c127d3d46cbd376: Status 404 returned error can't find the container with id 0102bee251e9376d4c51952774db0b44b4576d32060647984c127d3d46cbd376
Jul 07 14:26:27.370696 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.370655    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m2nzp"
Jul 07 14:26:27.378808 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:27.378780    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod324585d5_c3f9_4d3a_97b6_01755df3f0bd.slice/crio-3df63c315b90126367d0b69360de4063561105a9eef85925fe96da167375b6c4 WatchSource:0}: Error finding container 3df63c315b90126367d0b69360de4063561105a9eef85925fe96da167375b6c4: Status 404 returned error can't find the container with id 3df63c315b90126367d0b69360de4063561105a9eef85925fe96da167375b6c4
Jul 07 14:26:27.412203 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.412180    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:26:27.414775 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.414759    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9p6fd"
Jul 07 14:26:27.424202 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:27.424176    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c6df4a_e05d_4526_97da_de6c79fbe402.slice/crio-5cfa4ec677c4c065b083fe9db925c65c751db9ba17b06f99642d531c1675ae88 WatchSource:0}: Error finding container 5cfa4ec677c4c065b083fe9db925c65c751db9ba17b06f99642d531c1675ae88: Status 404 returned error can't find the container with id 5cfa4ec677c4c065b083fe9db925c65c751db9ba17b06f99642d531c1675ae88
Jul 07 14:26:27.460754 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.460727    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:26:27.470907 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:26:27.470884    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc39143b3_6865_46e6_971c_9c33afb34b45.slice/crio-0b3e85ea26b2c44173c28b32c5a6eec9a0b8eb0edef6ea51e03023a83e70ec88 WatchSource:0}: Error finding container 0b3e85ea26b2c44173c28b32c5a6eec9a0b8eb0edef6ea51e03023a83e70ec88: Status 404 returned error can't find the container with id 0b3e85ea26b2c44173c28b32c5a6eec9a0b8eb0edef6ea51e03023a83e70ec88
Jul 07 14:26:27.472698 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.472679    2583 kuberuntime_manager.go:1459] "Unhandled Error" err="container ovn-controller start failed in pod ovnkube-node-h577t_openshift-ovn-kubernetes(c39143b3-6865-46e6-971c-9c33afb34b45): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 07 14:26:27.473920 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.473900    2583 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-rbac-proxy-node start failed in pod ovnkube-node-h577t_openshift-ovn-kubernetes(c39143b3-6865-46e6-971c-9c33afb34b45): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 07 14:26:27.477062 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.477017    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-h577t" podUID="c39143b3-6865-46e6-971c-9c33afb34b45"
Jul 07 14:26:27.578745 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.578716    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:27.578896 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.578822    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:26:27.578896 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.578869    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret podName:7e360a2f-655d-464f-98e3-00096c75fbe9 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:28.578856727 +0000 UTC m=+3.041756646 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret") pod "global-pull-secret-syncer-mk6nm" (UID: "7e360a2f-655d-464f-98e3-00096c75fbe9") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:26:27.679226 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.679145    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs\") pod \"network-metrics-daemon-8sq9p\" (UID: \"3b53ae89-367b-4c2a-99f2-6bf60db0f958\") " pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:27.679351 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.679307    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:26:27.679401 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.679372    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs podName:3b53ae89-367b-4c2a-99f2-6bf60db0f958 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:28.679357586 +0000 UTC m=+3.142257506 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs") pod "network-metrics-daemon-8sq9p" (UID: "3b53ae89-367b-4c2a-99f2-6bf60db0f958") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:26:27.780596 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.780555    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwvdh\" (UniqueName: \"kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh\") pod \"network-check-target-gp4zb\" (UID: \"1c687b00-93cd-4401-a4c7-04aa5c966647\") " pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:27.780804 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.780788    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:26:27.780871 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.780810    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:26:27.780871 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.780823    2583 projected.go:196] Error preparing data for projected volume kube-api-access-pwvdh for pod openshift-network-diagnostics/network-check-target-gp4zb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:26:27.780990 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:27.780881    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh podName:1c687b00-93cd-4401-a4c7-04aa5c966647 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:28.780861243 +0000 UTC m=+3.243761167 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pwvdh" (UniqueName: "kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh") pod "network-check-target-gp4zb" (UID: "1c687b00-93cd-4401-a4c7-04aa5c966647") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:26:27.840217 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:27.840183    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:26:28.016226 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.015837    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:21:27 +0000 UTC" deadline="2028-04-02 17:37:36.459862816 +0000 UTC"
Jul 07 14:26:28.016226 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.015890    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15243h11m8.443976768s"
Jul 07 14:26:28.107552 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.107496    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-28k4c" event={"ID":"13805be4-13ec-42d1-a93d-b7fff2258a24","Type":"ContainerStarted","Data":"4374a043cf22c20a779e60cf95750ba629e760216ee9d6052c3f6b97827309a4"}
Jul 07 14:26:28.111536 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.111362    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8blpl" event={"ID":"3d0c2bd8-2635-4560-9823-91baebc88fbf","Type":"ContainerStarted","Data":"f5361544ae9011730399d50005c74cc2e53d4a6cd1b2e36ca762f27f5a8fb9b2"}
Jul 07 14:26:28.118890 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.118755    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal" event={"ID":"e20e813e5c8ec59bb9d94b29eb79c2ad","Type":"ContainerStarted","Data":"c7b14421cadb4be7ee447b77a6f4d5449d9c896ca0cf1b6f05f9b3b550fea665"}
Jul 07 14:26:28.131193 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.131151    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h577t" event={"ID":"c39143b3-6865-46e6-971c-9c33afb34b45","Type":"ContainerStarted","Data":"0b3e85ea26b2c44173c28b32c5a6eec9a0b8eb0edef6ea51e03023a83e70ec88"}
Jul 07 14:26:28.136248 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.136204    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9p6fd" event={"ID":"92c6df4a-e05d-4526-97da-de6c79fbe402","Type":"ContainerStarted","Data":"5cfa4ec677c4c065b083fe9db925c65c751db9ba17b06f99642d531c1675ae88"}
Jul 07 14:26:28.145376 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.145348    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m2nzp" event={"ID":"324585d5-c3f9-4d3a-97b6-01755df3f0bd","Type":"ContainerStarted","Data":"3df63c315b90126367d0b69360de4063561105a9eef85925fe96da167375b6c4"}
Jul 07 14:26:28.147147 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.147065    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287" event={"ID":"14321cfa-5c8b-408a-a66c-eb1c37d702e6","Type":"ContainerStarted","Data":"0102bee251e9376d4c51952774db0b44b4576d32060647984c127d3d46cbd376"}
Jul 07 14:26:28.147559 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:28.147497    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:52d26498fbce9f8ab56799e6e6f1920bc238d2d31f38945eab91ebc5f68b43dc\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:72c58cd87dea47bcff20bc07f2a9a5def3716e1a5b3d3e46ddcca8b49e9abb94\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-h577t" podUID="c39143b3-6865-46e6-971c-9c33afb34b45"
Jul 07 14:26:28.159404 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.159328    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g62rg" event={"ID":"e0def9a5-886b-490c-8a02-32e9f5dcecc5","Type":"ContainerStarted","Data":"b2c92af130e4b0260528f7ea935b768d964c662f67ead2958f3e592fe95f5c59"}
Jul 07 14:26:28.167648 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.167614    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5mt8" event={"ID":"c53d55ac-10e7-40fa-a2b5-e027292cda95","Type":"ContainerStarted","Data":"06bcf3f2ca62fe15ebbb17b8861c74fd339627e265ba1293edf9cebb3ec62520"}
Jul 07 14:26:28.170784 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.170753    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-1.ec2.internal" event={"ID":"603217e51a894b3722465edfd41817b7","Type":"ContainerStarted","Data":"daa1054fd8e52950b7bc97d24cd2b146c510f87c15462bbf8ab1efab237fe8d0"}
Jul 07 14:26:28.176630 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.176603    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sr65q" event={"ID":"06eab125-4856-449a-982f-111e9231f1d9","Type":"ContainerStarted","Data":"83a6d2a176aec8344b95e2051c7ceac96bf5466c12fa25d61e8336534e71976b"}
Jul 07 14:26:28.589155 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.589113    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:28.589351 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:28.589287    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:26:28.589412 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:28.589359    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret podName:7e360a2f-655d-464f-98e3-00096c75fbe9 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:30.58934081 +0000 UTC m=+5.052240733 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret") pod "global-pull-secret-syncer-mk6nm" (UID: "7e360a2f-655d-464f-98e3-00096c75fbe9") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:26:28.690204 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.690163    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs\") pod \"network-metrics-daemon-8sq9p\" (UID: \"3b53ae89-367b-4c2a-99f2-6bf60db0f958\") " pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:28.690387 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:28.690341    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:26:28.690441 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:28.690403    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs podName:3b53ae89-367b-4c2a-99f2-6bf60db0f958 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:30.690383474 +0000 UTC m=+5.153283397 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs") pod "network-metrics-daemon-8sq9p" (UID: "3b53ae89-367b-4c2a-99f2-6bf60db0f958") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:26:28.791233 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:28.791194    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwvdh\" (UniqueName: \"kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh\") pod \"network-check-target-gp4zb\" (UID: \"1c687b00-93cd-4401-a4c7-04aa5c966647\") " pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:28.791441 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:28.791417    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:26:28.791540 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:28.791445    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:26:28.791540 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:28.791458    2583 projected.go:196] Error preparing data for projected volume kube-api-access-pwvdh for pod openshift-network-diagnostics/network-check-target-gp4zb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:26:28.791540 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:28.791519    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh podName:1c687b00-93cd-4401-a4c7-04aa5c966647 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:30.791498899 +0000 UTC m=+5.254398821 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pwvdh" (UniqueName: "kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh") pod "network-check-target-gp4zb" (UID: "1c687b00-93cd-4401-a4c7-04aa5c966647") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:26:29.017057 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:29.016933    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:21:27 +0000 UTC" deadline="2028-04-20 17:01:52.927779234 +0000 UTC"
Jul 07 14:26:29.017057 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:29.017012    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15674h35m23.910770807s"
Jul 07 14:26:29.094648 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:29.094612    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:29.094835 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:29.094762    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:29.094892 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:29.094852    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:29.094939 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:29.094918    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:29.095017 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:29.095010    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:29.095114 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:29.095094    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:29.963491 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:29.963427    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:26:30.014088 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:30.013878    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 07 14:26:30.608365 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:30.608321    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:30.608799 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:30.608510    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:26:30.608799 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:30.608589    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret podName:7e360a2f-655d-464f-98e3-00096c75fbe9 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:34.608568167 +0000 UTC m=+9.071468091 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret") pod "global-pull-secret-syncer-mk6nm" (UID: "7e360a2f-655d-464f-98e3-00096c75fbe9") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:26:30.709409 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:30.709373    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs\") pod \"network-metrics-daemon-8sq9p\" (UID: \"3b53ae89-367b-4c2a-99f2-6bf60db0f958\") " pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:30.709605 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:30.709575    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:26:30.709671 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:30.709662    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs podName:3b53ae89-367b-4c2a-99f2-6bf60db0f958 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:34.709641428 +0000 UTC m=+9.172541352 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs") pod "network-metrics-daemon-8sq9p" (UID: "3b53ae89-367b-4c2a-99f2-6bf60db0f958") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:26:30.810395 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:30.810357    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwvdh\" (UniqueName: \"kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh\") pod \"network-check-target-gp4zb\" (UID: \"1c687b00-93cd-4401-a4c7-04aa5c966647\") " pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:30.810572 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:30.810523    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:26:30.810572 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:30.810546    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:26:30.810572 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:30.810557    2583 projected.go:196] Error preparing data for projected volume kube-api-access-pwvdh for pod openshift-network-diagnostics/network-check-target-gp4zb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:26:30.810730 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:30.810623    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh podName:1c687b00-93cd-4401-a4c7-04aa5c966647 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:34.810604383 +0000 UTC m=+9.273504305 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pwvdh" (UniqueName: "kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh") pod "network-check-target-gp4zb" (UID: "1c687b00-93cd-4401-a4c7-04aa5c966647") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:26:31.094946 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:31.094916    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:31.095126 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:31.094915    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:31.095126 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:31.094915    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:31.095126 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:31.095080    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:31.095285 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:31.095180    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:31.095285 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:31.095253    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:33.094709 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:33.094677    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:33.095259 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:33.094769    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:33.095259 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:33.095227    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:33.095259 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:33.095227    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:33.095395 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:33.095371    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:33.095851 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:33.095832    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:34.639703 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:34.639664    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:34.640137 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:34.639791    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:26:34.640137 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:34.639854    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret podName:7e360a2f-655d-464f-98e3-00096c75fbe9 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:42.639836905 +0000 UTC m=+17.102736829 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret") pod "global-pull-secret-syncer-mk6nm" (UID: "7e360a2f-655d-464f-98e3-00096c75fbe9") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:26:34.740232 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:34.740176    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs\") pod \"network-metrics-daemon-8sq9p\" (UID: \"3b53ae89-367b-4c2a-99f2-6bf60db0f958\") " pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:34.740400 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:34.740361    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:26:34.740457 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:34.740420    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs podName:3b53ae89-367b-4c2a-99f2-6bf60db0f958 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:42.740402457 +0000 UTC m=+17.203302381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs") pod "network-metrics-daemon-8sq9p" (UID: "3b53ae89-367b-4c2a-99f2-6bf60db0f958") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:26:34.841048 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:34.840944    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwvdh\" (UniqueName: \"kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh\") pod \"network-check-target-gp4zb\" (UID: \"1c687b00-93cd-4401-a4c7-04aa5c966647\") " pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:34.841211 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:34.841175    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:26:34.841211 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:34.841199    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:26:34.841211 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:34.841212    2583 projected.go:196] Error preparing data for projected volume kube-api-access-pwvdh for pod openshift-network-diagnostics/network-check-target-gp4zb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:26:34.841426 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:34.841270    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh podName:1c687b00-93cd-4401-a4c7-04aa5c966647 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:42.841250866 +0000 UTC m=+17.304150788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pwvdh" (UniqueName: "kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh") pod "network-check-target-gp4zb" (UID: "1c687b00-93cd-4401-a4c7-04aa5c966647") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:26:35.095305 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:35.095225    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:35.095470 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:35.095382    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:35.095470 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:35.095405    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:35.095470 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:35.095400    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:35.095638 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:35.095467    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:35.095638 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:35.095542    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:37.095581 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.095372    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:37.096075 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.095412    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:37.096075 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:37.095669    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:37.096075 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:37.095791    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:37.096075 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.095412    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:37.096075 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:37.095924    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:37.201638 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.201376    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m2nzp" event={"ID":"324585d5-c3f9-4d3a-97b6-01755df3f0bd","Type":"ContainerStarted","Data":"16ead17097fc837eddc7cd419ce747a9a4537015125a4cf3201da976032ca1b5"}
Jul 07 14:26:37.203207 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.203148    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287" event={"ID":"14321cfa-5c8b-408a-a66c-eb1c37d702e6","Type":"ContainerStarted","Data":"2346a560970dda71494165a59359516f4a139ad5ab2d1b4ab400fcc34e43b2e9"}
Jul 07 14:26:37.205121 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.205089    2583 generic.go:356] "Generic (PLEG): container finished" podID="c53d55ac-10e7-40fa-a2b5-e027292cda95" containerID="176fbb1e0164adc5ada8ef620ed981f612c0203b8fb6fb15eca121d9ca5698c4" exitCode=0
Jul 07 14:26:37.205360 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.205303    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5mt8" event={"ID":"c53d55ac-10e7-40fa-a2b5-e027292cda95","Type":"ContainerDied","Data":"176fbb1e0164adc5ada8ef620ed981f612c0203b8fb6fb15eca121d9ca5698c4"}
Jul 07 14:26:37.207722 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.207698    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-1.ec2.internal" event={"ID":"603217e51a894b3722465edfd41817b7","Type":"ContainerStarted","Data":"ad5408b5aa0d127fc63a1691524c210e81266d2b94694bdbac2c6172120bcefd"}
Jul 07 14:26:37.209436 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.209414    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sr65q" event={"ID":"06eab125-4856-449a-982f-111e9231f1d9","Type":"ContainerStarted","Data":"c2be9a5af27b506bc7b44a04402a22af343d43bf5f65ccf1f6e344112867605e"}
Jul 07 14:26:37.211355 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.211334    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-28k4c" event={"ID":"13805be4-13ec-42d1-a93d-b7fff2258a24","Type":"ContainerStarted","Data":"486c71f3e6365b457cbbc971dd6e9ce0f7200d78f179393adec5f88f85b36bd8"}
Jul 07 14:26:37.213490 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.213467    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8blpl" event={"ID":"3d0c2bd8-2635-4560-9823-91baebc88fbf","Type":"ContainerStarted","Data":"80574f12d24ae7d298e90d688027f6534e74f63daeb3b7753e933da1dfa60b92"}
Jul 07 14:26:37.215405 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.215167    2583 generic.go:356] "Generic (PLEG): container finished" podID="e20e813e5c8ec59bb9d94b29eb79c2ad" containerID="3498f06bda408ca7917faa7f66d55ed831beceb920a8aa2f7fea1bf190c55499" exitCode=0
Jul 07 14:26:37.215405 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.215205    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal" event={"ID":"e20e813e5c8ec59bb9d94b29eb79c2ad","Type":"ContainerDied","Data":"3498f06bda408ca7917faa7f66d55ed831beceb920a8aa2f7fea1bf190c55499"}
Jul 07 14:26:37.217963 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.217911    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-m2nzp" podStartSLOduration=2.349713443 podStartE2EDuration="11.217894852s" totalImagesPullingTime="8.868181409s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:37.217757721 +0000 UTC m=+11.680657655" watchObservedRunningTime="2026-07-07 14:26:37.217894852 +0000 UTC m=+11.680794796"
Jul 07 14:26:37.241644 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.241582    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8blpl" podStartSLOduration=2.251054274 podStartE2EDuration="11.24156123s" totalImagesPullingTime="8.990506956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:37.238528265 +0000 UTC m=+11.701428209" watchObservedRunningTime="2026-07-07 14:26:37.24156123 +0000 UTC m=+11.704461173"
Jul 07 14:26:37.299699 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.299402    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-1.ec2.internal" podStartSLOduration=11.299380526 podStartE2EDuration="11.299380526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:37.29895019 +0000 UTC m=+11.761850134" watchObservedRunningTime="2026-07-07 14:26:37.299380526 +0000 UTC m=+11.762280472"
Jul 07 14:26:37.315495 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.315429    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-28k4c" podStartSLOduration=2.377537986 podStartE2EDuration="11.315407548s" totalImagesPullingTime="8.937869562s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:37.314758011 +0000 UTC m=+11.777657955" watchObservedRunningTime="2026-07-07 14:26:37.315407548 +0000 UTC m=+11.778307490"
Jul 07 14:26:37.331194 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.330847    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sr65q" podStartSLOduration=2.444786376 podStartE2EDuration="11.330825464s" totalImagesPullingTime="8.886039088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:37.330736581 +0000 UTC m=+11.793636526" watchObservedRunningTime="2026-07-07 14:26:37.330825464 +0000 UTC m=+11.793725407"
Jul 07 14:26:37.597001 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:37.596963    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 07 14:26:38.017519 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:38.017299    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T14:26:37.596998154Z","UUID":"05bbe51e-20be-4052-a3ea-a6669fdbfa73","Handler":null,"Name":"","Endpoint":""}
Jul 07 14:26:38.019001 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:38.018966    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 07 14:26:38.019118 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:38.019016    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 07 14:26:38.220090 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:38.219823    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal" event={"ID":"e20e813e5c8ec59bb9d94b29eb79c2ad","Type":"ContainerStarted","Data":"07f18c4c045b17b7c7f9dd429f3f4469b04919d79c39f3fd079e9d6ab6bd6375"}
Jul 07 14:26:38.222351 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:38.222309    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287" event={"ID":"14321cfa-5c8b-408a-a66c-eb1c37d702e6","Type":"ContainerStarted","Data":"619db97007f0088979c61ea3d9ba8eb345ca5d493a4e174b19059d28bea464ab"}
Jul 07 14:26:38.224250 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:38.224213    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g62rg" event={"ID":"e0def9a5-886b-490c-8a02-32e9f5dcecc5","Type":"ContainerStarted","Data":"c6439fb76d6de7bd68041e1c5f742e16b6b5732d72522d11ad5e67d336d82686"}
Jul 07 14:26:38.240555 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:38.240504    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-1.ec2.internal" podStartSLOduration=12.240487749 podStartE2EDuration="12.240487749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:38.240270279 +0000 UTC m=+12.703170221" watchObservedRunningTime="2026-07-07 14:26:38.240487749 +0000 UTC m=+12.703387691"
Jul 07 14:26:38.255841 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:38.255779    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-g62rg" podStartSLOduration=3.331113736 podStartE2EDuration="12.255757867s" totalImagesPullingTime="8.924644131s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:38.255287291 +0000 UTC m=+12.718187235" watchObservedRunningTime="2026-07-07 14:26:38.255757867 +0000 UTC m=+12.718657809"
Jul 07 14:26:38.540666 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:38.540624    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sr65q"
Jul 07 14:26:38.541230 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:38.541205    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sr65q"
Jul 07 14:26:39.095156 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:39.095124    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:39.095343 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:39.095233    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:39.095343 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:39.095269    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:39.095459 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:39.095342    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:39.095459 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:39.095241    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:39.095561 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:39.095511    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:40.098625 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:40.098598    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:40.099203 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:40.098703    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:40.228540 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:40.228510    2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 07 14:26:41.094571 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:41.094536    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:41.094571 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:41.094576    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:41.094842 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:41.094692    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:41.094842 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:41.094826    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:42.101292 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:42.101259    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:42.102003 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:42.101377    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:42.707029 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:42.706747    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:42.707224 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:42.706940    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:26:42.707224 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:42.707139    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret podName:7e360a2f-655d-464f-98e3-00096c75fbe9 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:58.707116719 +0000 UTC m=+33.170016715 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret") pod "global-pull-secret-syncer-mk6nm" (UID: "7e360a2f-655d-464f-98e3-00096c75fbe9") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:26:42.807563 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:42.807506    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs\") pod \"network-metrics-daemon-8sq9p\" (UID: \"3b53ae89-367b-4c2a-99f2-6bf60db0f958\") " pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:42.807835 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:42.807713    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:26:42.807835 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:42.807763    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs podName:3b53ae89-367b-4c2a-99f2-6bf60db0f958 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:58.807748681 +0000 UTC m=+33.270648601 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs") pod "network-metrics-daemon-8sq9p" (UID: "3b53ae89-367b-4c2a-99f2-6bf60db0f958") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:26:42.908909 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:42.908869    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwvdh\" (UniqueName: \"kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh\") pod \"network-check-target-gp4zb\" (UID: \"1c687b00-93cd-4401-a4c7-04aa5c966647\") " pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:42.909083 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:42.909051    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:26:42.909083 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:42.909072    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:26:42.909161 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:42.909087    2583 projected.go:196] Error preparing data for projected volume kube-api-access-pwvdh for pod openshift-network-diagnostics/network-check-target-gp4zb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:26:42.909161 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:42.909145    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh podName:1c687b00-93cd-4401-a4c7-04aa5c966647 nodeName:}" failed. No retries permitted until 2026-07-07 14:26:58.90912623 +0000 UTC m=+33.372026152 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pwvdh" (UniqueName: "kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh") pod "network-check-target-gp4zb" (UID: "1c687b00-93cd-4401-a4c7-04aa5c966647") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:26:43.095199 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:43.095122    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:43.095363 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:43.095247    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:43.095442 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:43.095126    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:43.095595 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:43.095570    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:43.238252 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:43.238212    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287" event={"ID":"14321cfa-5c8b-408a-a66c-eb1c37d702e6","Type":"ContainerStarted","Data":"5450987e4337edfd23e7516f73f3120c8eb1675dc4ee640e3a07ef1bec17b8d5"}
Jul 07 14:26:43.256422 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:43.256365    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dn287" podStartSLOduration=2.817046438 podStartE2EDuration="17.256343719s" totalImagesPullingTime="14.439297281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:43.25576522 +0000 UTC m=+17.718665164" watchObservedRunningTime="2026-07-07 14:26:43.256343719 +0000 UTC m=+17.719243663"
Jul 07 14:26:44.097856 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:44.097830    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:44.098031 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:44.097946    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:45.095475 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:45.095438    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:45.095935 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:45.095575    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:45.095935 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:45.095442    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:45.096082 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:45.095966    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:46.097691 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:46.097664    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:46.098051 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:46.097755    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:46.246242 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:46.246184    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9p6fd" event={"ID":"92c6df4a-e05d-4526-97da-de6c79fbe402","Type":"ContainerStarted","Data":"1bc659359c37ed87e3f6da6ca5a20d9480030350c0cfd86424d2d92890071438"}
Jul 07 14:26:46.248193 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:46.248167    2583 generic.go:356] "Generic (PLEG): container finished" podID="c53d55ac-10e7-40fa-a2b5-e027292cda95" containerID="a33b940756ce67a6919577f06a20d8d1a838e60f3a4fbeb6a65c30150e2955b2" exitCode=0
Jul 07 14:26:46.248340 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:46.248210    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5mt8" event={"ID":"c53d55ac-10e7-40fa-a2b5-e027292cda95","Type":"ContainerDied","Data":"a33b940756ce67a6919577f06a20d8d1a838e60f3a4fbeb6a65c30150e2955b2"}
Jul 07 14:26:46.262644 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:46.262599    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9p6fd" podStartSLOduration=2.432746627 podStartE2EDuration="20.262586222s" totalImagesPullingTime="17.829839595s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:26:46.262320198 +0000 UTC m=+20.725220142" watchObservedRunningTime="2026-07-07 14:26:46.262586222 +0000 UTC m=+20.725486167"
Jul 07 14:26:47.094841 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:47.094805    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:47.095048 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:47.094937    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:47.095048 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:47.094817    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:47.095177 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:47.095126    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:48.099932 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:48.099906    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:48.100318 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:48.100051    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:48.254117 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:48.254081    2583 generic.go:356] "Generic (PLEG): container finished" podID="c53d55ac-10e7-40fa-a2b5-e027292cda95" containerID="1d8d01d032155185acf7a6fc1ed4927d5b12aa45600d5df9ac85cabc716074f8" exitCode=0
Jul 07 14:26:48.254301 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:48.254138    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5mt8" event={"ID":"c53d55ac-10e7-40fa-a2b5-e027292cda95","Type":"ContainerDied","Data":"1d8d01d032155185acf7a6fc1ed4927d5b12aa45600d5df9ac85cabc716074f8"}
Jul 07 14:26:49.095477 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:49.095444    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:49.095619 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:49.095448    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:49.095619 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:49.095573    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:49.095728 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:49.095658    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:49.258328 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:49.258240    2583 generic.go:356] "Generic (PLEG): container finished" podID="c53d55ac-10e7-40fa-a2b5-e027292cda95" containerID="27aae814e5a429401a11dfe170d10d45fc21239a0eed38fb35c000bab1071542" exitCode=0
Jul 07 14:26:49.258328 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:49.258292    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5mt8" event={"ID":"c53d55ac-10e7-40fa-a2b5-e027292cda95","Type":"ContainerDied","Data":"27aae814e5a429401a11dfe170d10d45fc21239a0eed38fb35c000bab1071542"}
Jul 07 14:26:50.096140 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:50.095426    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:50.096140 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:50.095705    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:51.094911 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:51.094876    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:51.095490 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:51.094922    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:51.095490 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:51.095052    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:51.095490 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:51.095175    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:52.095447 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:52.095414    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:52.095911 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:52.095541    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:53.094763 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:53.094722    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:53.094763 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:53.094754    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:53.095029 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:53.094875    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:53.095071 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:53.095023    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:54.095089 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:54.095051    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:54.095679 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:54.095172    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:55.095430 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:55.095395    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:55.095430 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:55.095414    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:55.095951 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:55.095521    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:55.095951 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:55.095641    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:56.095989 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:56.095936    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:56.096461 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:56.096071    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:56.399396 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:56.399318    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sr65q"
Jul 07 14:26:56.399564 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:56.399475    2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 07 14:26:56.399908 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:56.399859    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sr65q"
Jul 07 14:26:57.095128 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:57.095093    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:57.095291 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:57.095093    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:57.095291 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:57.095195    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:57.095291 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:57.095268    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:58.095098 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:58.094910    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:58.095539 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:58.095181    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:26:58.285456 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:58.285429    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:26:58.285823 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:58.285794    2583 generic.go:356] "Generic (PLEG): container finished" podID="c39143b3-6865-46e6-971c-9c33afb34b45" containerID="3de70c7bbf2934706b0ea1bb6b9fd1e378ba7dfd6a2860eae778c1379f807104" exitCode=1
Jul 07 14:26:58.285935 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:58.285867    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h577t" event={"ID":"c39143b3-6865-46e6-971c-9c33afb34b45","Type":"ContainerStarted","Data":"815376cc0cea054bb54b199edba69daa20da934a57cb30f15eac8ffd1ba4c503"}
Jul 07 14:26:58.286026 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:58.285951    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h577t" event={"ID":"c39143b3-6865-46e6-971c-9c33afb34b45","Type":"ContainerStarted","Data":"dd1947a2ff8d9b6ce4014af7331b1d1e4b30746d1c2d0acbac3d086a5f9b9db4"}
Jul 07 14:26:58.286026 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:58.286013    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h577t" event={"ID":"c39143b3-6865-46e6-971c-9c33afb34b45","Type":"ContainerStarted","Data":"f60a11f68fbd5dc2da40cb6f9d05b01bdd5a719afb9a19c8c549c596c0f1602e"}
Jul 07 14:26:58.286116 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:58.286044    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h577t" event={"ID":"c39143b3-6865-46e6-971c-9c33afb34b45","Type":"ContainerStarted","Data":"36dda2072e28d95ae4d74576de3de8dff237373e09b9462117df09d682069e17"}
Jul 07 14:26:58.286116 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:58.286071    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h577t" event={"ID":"c39143b3-6865-46e6-971c-9c33afb34b45","Type":"ContainerDied","Data":"3de70c7bbf2934706b0ea1bb6b9fd1e378ba7dfd6a2860eae778c1379f807104"}
Jul 07 14:26:58.286116 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:58.286106    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h577t" event={"ID":"c39143b3-6865-46e6-971c-9c33afb34b45","Type":"ContainerStarted","Data":"5dfc6e0097cad139ca0a3af8ee8b387681d631e4f6e75068adc506b6b8c1d1b3"}
Jul 07 14:26:58.288627 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:58.288606    2583 generic.go:356] "Generic (PLEG): container finished" podID="c53d55ac-10e7-40fa-a2b5-e027292cda95" containerID="fa1bab0d905c8cc6d887db1a559258b47167b6b4ce7c8d391875f6aaa9770449" exitCode=0
Jul 07 14:26:58.288727 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:58.288641    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5mt8" event={"ID":"c53d55ac-10e7-40fa-a2b5-e027292cda95","Type":"ContainerDied","Data":"fa1bab0d905c8cc6d887db1a559258b47167b6b4ce7c8d391875f6aaa9770449"}
Jul 07 14:26:58.728121 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:58.728088    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:26:58.728293 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:58.728204    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 07 14:26:58.728293 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:58.728257    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret podName:7e360a2f-655d-464f-98e3-00096c75fbe9 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:30.728243917 +0000 UTC m=+65.191143837 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret") pod "global-pull-secret-syncer-mk6nm" (UID: "7e360a2f-655d-464f-98e3-00096c75fbe9") : object "kube-system"/"original-pull-secret" not registered
Jul 07 14:26:58.828768 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:58.828738    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs\") pod \"network-metrics-daemon-8sq9p\" (UID: \"3b53ae89-367b-4c2a-99f2-6bf60db0f958\") " pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:58.828941 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:58.828849    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:26:58.828941 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:58.828899    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs podName:3b53ae89-367b-4c2a-99f2-6bf60db0f958 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:30.828885174 +0000 UTC m=+65.291785093 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs") pod "network-metrics-daemon-8sq9p" (UID: "3b53ae89-367b-4c2a-99f2-6bf60db0f958") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 07 14:26:58.930069 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:58.930033    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwvdh\" (UniqueName: \"kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh\") pod \"network-check-target-gp4zb\" (UID: \"1c687b00-93cd-4401-a4c7-04aa5c966647\") " pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:58.930238 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:58.930183    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 07 14:26:58.930238 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:58.930200    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 07 14:26:58.930238 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:58.930212    2583 projected.go:196] Error preparing data for projected volume kube-api-access-pwvdh for pod openshift-network-diagnostics/network-check-target-gp4zb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:26:58.930333 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:58.930260    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh podName:1c687b00-93cd-4401-a4c7-04aa5c966647 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:30.930246619 +0000 UTC m=+65.393146539 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pwvdh" (UniqueName: "kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh") pod "network-check-target-gp4zb" (UID: "1c687b00-93cd-4401-a4c7-04aa5c966647") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 07 14:26:59.095533 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:59.095451    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:26:59.095533 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:59.095489    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:26:59.096038 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:59.095565    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:26:59.096038 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:26:59.095683    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:26:59.294159 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:59.294124    2583 generic.go:356] "Generic (PLEG): container finished" podID="c53d55ac-10e7-40fa-a2b5-e027292cda95" containerID="95747fcf1aecf5b976c934645121ba7fc547df54b2b543f7165df60a7148f4eb" exitCode=0
Jul 07 14:26:59.294305 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:26:59.294171    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5mt8" event={"ID":"c53d55ac-10e7-40fa-a2b5-e027292cda95","Type":"ContainerDied","Data":"95747fcf1aecf5b976c934645121ba7fc547df54b2b543f7165df60a7148f4eb"}
Jul 07 14:27:00.094998 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:00.094943    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:27:00.095168 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:00.095076    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:27:00.299401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:00.299372    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k5mt8" event={"ID":"c53d55ac-10e7-40fa-a2b5-e027292cda95","Type":"ContainerStarted","Data":"58fe2c29a176c32d3084ba3330212d69ff0e8f7d40556d47ddb73c9b7851a5fa"}
Jul 07 14:27:00.324396 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:00.324334    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-k5mt8" podStartSLOduration=7.402470408 podStartE2EDuration="34.324311119s" totalImagesPullingTime="26.921840711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:00.323295802 +0000 UTC m=+34.786195743" watchObservedRunningTime="2026-07-07 14:27:00.324311119 +0000 UTC m=+34.787211062"
Jul 07 14:27:01.094764 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:01.094724    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:27:01.094935 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:01.094861    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:27:01.094935 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:01.094904    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:27:01.095164 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:01.095004    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:27:01.304597 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:01.304569    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:27:01.305124 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:01.304996    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h577t" event={"ID":"c39143b3-6865-46e6-971c-9c33afb34b45","Type":"ContainerStarted","Data":"c5eae87b0304dd2a033e516981619e54eeee8734b5d6e4696be2f0145195ee67"}
Jul 07 14:27:02.095469 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:02.095438    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:27:02.095648 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:02.095541    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:27:03.095640 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:03.095459    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:27:03.096026 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:03.095501    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:27:03.096026 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:03.095724    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:27:03.096026 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:03.095770    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:27:03.106870 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:03.106848    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m2nzp_324585d5-c3f9-4d3a-97b6-01755df3f0bd/dns-node-resolver/0.log"
Jul 07 14:27:03.312859 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:03.312830    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:27:03.313211 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:03.313190    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h577t" event={"ID":"c39143b3-6865-46e6-971c-9c33afb34b45","Type":"ContainerStarted","Data":"5901117866b9b7955f5da2517dd6a4f670257f04a36469be5a50e99d0598b6d2"}
Jul 07 14:27:03.313484 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:03.313468    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:27:03.313691 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:03.313669    2583 scope.go:117] "RemoveContainer" containerID="3de70c7bbf2934706b0ea1bb6b9fd1e378ba7dfd6a2860eae778c1379f807104"
Jul 07 14:27:03.330307 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:03.330286    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:27:03.685068 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:03.685046    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-28k4c_13805be4-13ec-42d1-a93d-b7fff2258a24/node-ca/0.log"
Jul 07 14:27:04.094961 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:04.094879    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:27:04.095138 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:04.095020    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:27:04.318892 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:04.318861    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:27:04.319374 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:04.319260    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h577t" event={"ID":"c39143b3-6865-46e6-971c-9c33afb34b45","Type":"ContainerStarted","Data":"75d3796036fd43087c8182a058991f9306fbaaa1365ff126ca5f61edb8edd217"}
Jul 07 14:27:04.319441 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:04.319424    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:27:04.319503 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:04.319487    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:27:04.335463 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:04.335432    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:27:04.352010 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:04.351904    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h577t" podStartSLOduration=8.268356276 podStartE2EDuration="38.351887442s" totalImagesPullingTime="30.083531166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=2 observedRunningTime="2026-07-07 14:27:04.351700954 +0000 UTC m=+38.814600906" watchObservedRunningTime="2026-07-07 14:27:04.351887442 +0000 UTC m=+38.814787383"
Jul 07 14:27:04.478274 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:04.478247    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gp4zb"]
Jul 07 14:27:04.478442 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:04.478345    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:27:04.478442 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:04.478422    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:27:04.480770 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:04.480746    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8sq9p"]
Jul 07 14:27:04.480872 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:04.480839    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:27:04.480964 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:04.480943    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:27:04.492354 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:04.492329    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mk6nm"]
Jul 07 14:27:04.492460 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:04.492403    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:27:04.492534 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:04.492471    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:27:06.096447 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:06.096412    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:27:06.096988 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:06.096501    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:27:06.096988 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:06.096548    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:27:06.096988 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:06.096569    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:27:06.096988 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:06.096624    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:27:06.096988 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:06.096685    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:27:08.095476 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:08.095283    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:27:08.095896 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:08.095287    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:27:08.095896 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:08.095561    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gp4zb" podUID="1c687b00-93cd-4401-a4c7-04aa5c966647"
Jul 07 14:27:08.095896 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:08.095335    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:27:08.095896 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:08.095636    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8sq9p" podUID="3b53ae89-367b-4c2a-99f2-6bf60db0f958"
Jul 07 14:27:08.095896 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:08.095738    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mk6nm" podUID="7e360a2f-655d-464f-98e3-00096c75fbe9"
Jul 07 14:27:09.860339 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.860308    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-1.ec2.internal" event="NodeReady"
Jul 07 14:27:09.860688 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.860444    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 07 14:27:09.907108 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.907032    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nqmhq"]
Jul 07 14:27:09.951752 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.951718    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9tpzt"]
Jul 07 14:27:09.951915 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.951840    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:09.954470 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.954436    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 07 14:27:09.954603 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.954470    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fh5qh\""
Jul 07 14:27:09.954603 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.954521    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 07 14:27:09.954603 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.954536    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 07 14:27:09.954603 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.954562    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 07 14:27:09.973028 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.973002    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nqmhq"]
Jul 07 14:27:09.973145 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.973100    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9tpzt"
Jul 07 14:27:09.973145 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.973137    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9tpzt"]
Jul 07 14:27:09.975063 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.975038    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 07 14:27:09.975063 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.975054    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ck6xv\""
Jul 07 14:27:09.975063 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.975038    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 07 14:27:09.975274 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:09.975140    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 07 14:27:10.005048 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.005023    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-95td9"]
Jul 07 14:27:10.009802 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.009788    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:10.012015 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.011996    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 07 14:27:10.012139 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.012120    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 07 14:27:10.012340 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.012324    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ksq8k\""
Jul 07 14:27:10.017752 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.017731    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-95td9"]
Jul 07 14:27:10.095319 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.095283    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:27:10.095319 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.095324    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:27:10.095526 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.095286    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:27:10.099850 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.099825    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 14:27:10.100028 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.099865    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-29x7z\""
Jul 07 14:27:10.100028 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.099883    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 14:27:10.100028 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.099883    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 14:27:10.100028 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.099941    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ps6h4\""
Jul 07 14:27:10.100252 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.100077    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 14:27:10.112644 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.112621    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc038d77-b57f-4354-a128-672f0d0b5319-data-volume\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:10.112764 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.112647    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:10.112764 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.112662    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-tmp-dir\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:10.112764 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.112682    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/160fe77b-4e72-4855-9eb0-ac0b2708177c-cert\") pod \"ingress-canary-9tpzt\" (UID: \"160fe77b-4e72-4855-9eb0-ac0b2708177c\") " pod="openshift-ingress-canary/ingress-canary-9tpzt"
Jul 07 14:27:10.112764 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.112708    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fc038d77-b57f-4354-a128-672f0d0b5319-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:10.112764 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.112746    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfp5j\" (UniqueName: \"kubernetes.io/projected/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-kube-api-access-vfp5j\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:10.112926 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.112824    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzrkg\" (UniqueName: \"kubernetes.io/projected/fc038d77-b57f-4354-a128-672f0d0b5319-kube-api-access-dzrkg\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:10.112926 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.112871    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fc038d77-b57f-4354-a128-672f0d0b5319-crio-socket\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:10.112926 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.112889    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h9sl\" (UniqueName: \"kubernetes.io/projected/160fe77b-4e72-4855-9eb0-ac0b2708177c-kube-api-access-7h9sl\") pod \"ingress-canary-9tpzt\" (UID: \"160fe77b-4e72-4855-9eb0-ac0b2708177c\") " pod="openshift-ingress-canary/ingress-canary-9tpzt"
Jul 07 14:27:10.112926 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.112913    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:10.113083 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.112929    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-config-volume\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:10.213964 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.213937    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:10.214170 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.213971    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-config-volume\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:10.214170 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.214103    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc038d77-b57f-4354-a128-672f0d0b5319-data-volume\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:10.214170 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:10.214109    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:27:10.214170 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.214123    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:10.214170 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.214138    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-tmp-dir\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:10.214404 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:10.214185    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls podName:fc038d77-b57f-4354-a128-672f0d0b5319 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:10.714163996 +0000 UTC m=+45.177063929 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nqmhq" (UID: "fc038d77-b57f-4354-a128-672f0d0b5319") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:27:10.214404 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.214223    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/160fe77b-4e72-4855-9eb0-ac0b2708177c-cert\") pod \"ingress-canary-9tpzt\" (UID: \"160fe77b-4e72-4855-9eb0-ac0b2708177c\") " pod="openshift-ingress-canary/ingress-canary-9tpzt"
Jul 07 14:27:10.214404 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:10.214241    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:27:10.214404 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.214263    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fc038d77-b57f-4354-a128-672f0d0b5319-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:10.214404 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.214290    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfp5j\" (UniqueName: \"kubernetes.io/projected/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-kube-api-access-vfp5j\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:10.214404 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:10.214321    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls podName:fdf28b9f-131a-4f2a-94b0-a101c082c4b1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:10.714302507 +0000 UTC m=+45.177202428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls") pod "dns-default-95td9" (UID: "fdf28b9f-131a-4f2a-94b0-a101c082c4b1") : secret "dns-default-metrics-tls" not found
Jul 07 14:27:10.214404 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:10.214347    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:27:10.214404 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.214350    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzrkg\" (UniqueName: \"kubernetes.io/projected/fc038d77-b57f-4354-a128-672f0d0b5319-kube-api-access-dzrkg\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:10.214404 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:10.214388    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc038d77-b57f-4354-a128-672f0d0b5319-kube-rbac-proxy-cm podName:fc038d77-b57f-4354-a128-672f0d0b5319 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:10.714378884 +0000 UTC m=+45.177278804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fc038d77-b57f-4354-a128-672f0d0b5319-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nqmhq" (UID: "fc038d77-b57f-4354-a128-672f0d0b5319") : configmap "kube-rbac-proxy" not found
Jul 07 14:27:10.214842 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.214439    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fc038d77-b57f-4354-a128-672f0d0b5319-crio-socket\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:10.214842 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.214488    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-tmp-dir\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:10.214842 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.214510    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fc038d77-b57f-4354-a128-672f0d0b5319-data-volume\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:10.214842 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.214584    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h9sl\" (UniqueName: \"kubernetes.io/projected/160fe77b-4e72-4855-9eb0-ac0b2708177c-kube-api-access-7h9sl\") pod \"ingress-canary-9tpzt\" (UID: \"160fe77b-4e72-4855-9eb0-ac0b2708177c\") " pod="openshift-ingress-canary/ingress-canary-9tpzt"
Jul 07 14:27:10.214842 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.214652    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fc038d77-b57f-4354-a128-672f0d0b5319-crio-socket\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:10.214842 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.214812    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-config-volume\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:10.218233 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.218218    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/160fe77b-4e72-4855-9eb0-ac0b2708177c-cert\") pod \"ingress-canary-9tpzt\" (UID: \"160fe77b-4e72-4855-9eb0-ac0b2708177c\") " pod="openshift-ingress-canary/ingress-canary-9tpzt"
Jul 07 14:27:10.222551 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.222531    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfp5j\" (UniqueName: \"kubernetes.io/projected/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-kube-api-access-vfp5j\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:10.222881 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.222862    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h9sl\" (UniqueName: \"kubernetes.io/projected/160fe77b-4e72-4855-9eb0-ac0b2708177c-kube-api-access-7h9sl\") pod \"ingress-canary-9tpzt\" (UID: \"160fe77b-4e72-4855-9eb0-ac0b2708177c\") " pod="openshift-ingress-canary/ingress-canary-9tpzt"
Jul 07 14:27:10.224034 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.224014    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzrkg\" (UniqueName: \"kubernetes.io/projected/fc038d77-b57f-4354-a128-672f0d0b5319-kube-api-access-dzrkg\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:10.281240 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.281203    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9tpzt"
Jul 07 14:27:10.505326 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.505245    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9tpzt"]
Jul 07 14:27:10.508345 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:27:10.508319    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod160fe77b_4e72_4855_9eb0_ac0b2708177c.slice/crio-4e73e2a96a0b852f9be16b2f7984c6a27742f95027beee48c3d26b42cfe96708 WatchSource:0}: Error finding container 4e73e2a96a0b852f9be16b2f7984c6a27742f95027beee48c3d26b42cfe96708: Status 404 returned error can't find the container with id 4e73e2a96a0b852f9be16b2f7984c6a27742f95027beee48c3d26b42cfe96708
Jul 07 14:27:10.719656 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.719619    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:10.719816 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.719670    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fc038d77-b57f-4354-a128-672f0d0b5319-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:10.719816 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:10.719740    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:10.719816 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:10.719782    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:27:10.719993 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:10.719821    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:27:10.719993 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:10.719846    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls podName:fdf28b9f-131a-4f2a-94b0-a101c082c4b1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:11.719827876 +0000 UTC m=+46.182727815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls") pod "dns-default-95td9" (UID: "fdf28b9f-131a-4f2a-94b0-a101c082c4b1") : secret "dns-default-metrics-tls" not found
Jul 07 14:27:10.719993 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:10.719861    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:27:10.719993 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:10.719889    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc038d77-b57f-4354-a128-672f0d0b5319-kube-rbac-proxy-cm podName:fc038d77-b57f-4354-a128-672f0d0b5319 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:11.719874329 +0000 UTC m=+46.182774253 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fc038d77-b57f-4354-a128-672f0d0b5319-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nqmhq" (UID: "fc038d77-b57f-4354-a128-672f0d0b5319") : configmap "kube-rbac-proxy" not found
Jul 07 14:27:10.719993 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:10.719905    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls podName:fc038d77-b57f-4354-a128-672f0d0b5319 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:11.71989527 +0000 UTC m=+46.182795206 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nqmhq" (UID: "fc038d77-b57f-4354-a128-672f0d0b5319") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:27:11.336876 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:11.336836    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9tpzt" event={"ID":"160fe77b-4e72-4855-9eb0-ac0b2708177c","Type":"ContainerStarted","Data":"4e73e2a96a0b852f9be16b2f7984c6a27742f95027beee48c3d26b42cfe96708"}
Jul 07 14:27:11.727236 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:11.727200    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:11.727421 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:11.727251    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fc038d77-b57f-4354-a128-672f0d0b5319-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:11.727421 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:11.727317    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:27:11.727421 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:11.727382    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:27:11.727421 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:11.727321    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:11.727421 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:11.727395    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls podName:fdf28b9f-131a-4f2a-94b0-a101c082c4b1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:13.727372052 +0000 UTC m=+48.190271986 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls") pod "dns-default-95td9" (UID: "fdf28b9f-131a-4f2a-94b0-a101c082c4b1") : secret "dns-default-metrics-tls" not found
Jul 07 14:27:11.727672 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:11.727439    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:27:11.727672 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:11.727447    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc038d77-b57f-4354-a128-672f0d0b5319-kube-rbac-proxy-cm podName:fc038d77-b57f-4354-a128-672f0d0b5319 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:13.727431183 +0000 UTC m=+48.190331105 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fc038d77-b57f-4354-a128-672f0d0b5319-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nqmhq" (UID: "fc038d77-b57f-4354-a128-672f0d0b5319") : configmap "kube-rbac-proxy" not found
Jul 07 14:27:11.727672 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:11.727505    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls podName:fc038d77-b57f-4354-a128-672f0d0b5319 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:13.727489946 +0000 UTC m=+48.190389881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nqmhq" (UID: "fc038d77-b57f-4354-a128-672f0d0b5319") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:27:13.744922 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:13.744883    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:13.745404 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:13.744937    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:13.745404 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:13.744963    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fc038d77-b57f-4354-a128-672f0d0b5319-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:13.745404 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:13.745084    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:27:13.745404 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:13.745100    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 07 14:27:13.745404 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:13.745090    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:27:13.745404 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:13.745153    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fc038d77-b57f-4354-a128-672f0d0b5319-kube-rbac-proxy-cm podName:fc038d77-b57f-4354-a128-672f0d0b5319 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:17.745140343 +0000 UTC m=+52.208040265 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fc038d77-b57f-4354-a128-672f0d0b5319-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nqmhq" (UID: "fc038d77-b57f-4354-a128-672f0d0b5319") : configmap "kube-rbac-proxy" not found
Jul 07 14:27:13.745404 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:13.745168    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls podName:fdf28b9f-131a-4f2a-94b0-a101c082c4b1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:17.745162013 +0000 UTC m=+52.208061932 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls") pod "dns-default-95td9" (UID: "fdf28b9f-131a-4f2a-94b0-a101c082c4b1") : secret "dns-default-metrics-tls" not found
Jul 07 14:27:13.745404 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:13.745180    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls podName:fc038d77-b57f-4354-a128-672f0d0b5319 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:17.745174976 +0000 UTC m=+52.208074896 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nqmhq" (UID: "fc038d77-b57f-4354-a128-672f0d0b5319") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:27:14.345299 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:14.345266    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9tpzt" event={"ID":"160fe77b-4e72-4855-9eb0-ac0b2708177c","Type":"ContainerStarted","Data":"903a467df11ad35d647c7f61431c5aec1917504e3a8bf919906d42f031044cd5"}
Jul 07 14:27:14.362035 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:14.361966    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9tpzt" podStartSLOduration=2.502895198 podStartE2EDuration="5.361952276s" totalImagesPullingTime="2.859057078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:14.361348094 +0000 UTC m=+48.824248040" watchObservedRunningTime="2026-07-07 14:27:14.361952276 +0000 UTC m=+48.824852258"
Jul 07 14:27:16.749947 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:16.749916    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-8qstb"]
Jul 07 14:27:16.752795 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:16.752778    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8qstb"
Jul 07 14:27:16.754788 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:16.754767    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 07 14:27:16.754899 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:16.754767    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 07 14:27:16.754899 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:16.754839    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-nsx9z\""
Jul 07 14:27:16.761994 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:16.761953    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-8qstb"]
Jul 07 14:27:16.770601 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:16.770575    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhzdc\" (UniqueName: \"kubernetes.io/projected/aa4cebf1-cb91-4bd2-96a1-235bb4ebb3a4-kube-api-access-qhzdc\") pod \"migrator-d57d58cc9-8qstb\" (UID: \"aa4cebf1-cb91-4bd2-96a1-235bb4ebb3a4\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8qstb"
Jul 07 14:27:16.871812 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:16.871768    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhzdc\" (UniqueName: \"kubernetes.io/projected/aa4cebf1-cb91-4bd2-96a1-235bb4ebb3a4-kube-api-access-qhzdc\") pod \"migrator-d57d58cc9-8qstb\" (UID: \"aa4cebf1-cb91-4bd2-96a1-235bb4ebb3a4\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8qstb"
Jul 07 14:27:16.880327 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:16.880295    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhzdc\" (UniqueName: \"kubernetes.io/projected/aa4cebf1-cb91-4bd2-96a1-235bb4ebb3a4-kube-api-access-qhzdc\") pod \"migrator-d57d58cc9-8qstb\" (UID: \"aa4cebf1-cb91-4bd2-96a1-235bb4ebb3a4\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8qstb"
Jul 07 14:27:17.061756 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:17.061675    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8qstb"
Jul 07 14:27:17.226765 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:17.226734    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-8qstb"]
Jul 07 14:27:17.229339 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:27:17.229309    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa4cebf1_cb91_4bd2_96a1_235bb4ebb3a4.slice/crio-9cd1d6fc6428aeab14c28938cfa125d38dbd33040ad65dbf342fa08df4ff2ff4 WatchSource:0}: Error finding container 9cd1d6fc6428aeab14c28938cfa125d38dbd33040ad65dbf342fa08df4ff2ff4: Status 404 returned error can't find the container with id 9cd1d6fc6428aeab14c28938cfa125d38dbd33040ad65dbf342fa08df4ff2ff4
Jul 07 14:27:17.352607 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:17.352524    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8qstb" event={"ID":"aa4cebf1-cb91-4bd2-96a1-235bb4ebb3a4","Type":"ContainerStarted","Data":"9cd1d6fc6428aeab14c28938cfa125d38dbd33040ad65dbf342fa08df4ff2ff4"}
Jul 07 14:27:17.779472 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:17.779431    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:17.779850 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:17.779482    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fc038d77-b57f-4354-a128-672f0d0b5319-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:17.779850 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:17.779527    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:17.779850 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:17.779610    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:27:17.779850 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:17.779664    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls podName:fc038d77-b57f-4354-a128-672f0d0b5319 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:25.779649951 +0000 UTC m=+60.242549870 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nqmhq" (UID: "fc038d77-b57f-4354-a128-672f0d0b5319") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:27:17.779850 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:17.779609    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:27:17.779850 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:17.779779    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls podName:fdf28b9f-131a-4f2a-94b0-a101c082c4b1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:25.779765265 +0000 UTC m=+60.242665185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls") pod "dns-default-95td9" (UID: "fdf28b9f-131a-4f2a-94b0-a101c082c4b1") : secret "dns-default-metrics-tls" not found
Jul 07 14:27:17.780128 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:17.780056    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fc038d77-b57f-4354-a128-672f0d0b5319-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:19.359646 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:19.359604    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8qstb" event={"ID":"aa4cebf1-cb91-4bd2-96a1-235bb4ebb3a4","Type":"ContainerStarted","Data":"9795ee5c93076db05716aa4d5a7ef06f646064d14795220dea44ab805d9a3047"}
Jul 07 14:27:19.360044 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:19.359675    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8qstb" event={"ID":"aa4cebf1-cb91-4bd2-96a1-235bb4ebb3a4","Type":"ContainerStarted","Data":"bfd83d39bd29b2450db6627049d8356919ca3b09a2d4e2caea23821d242ae4fc"}
Jul 07 14:27:19.379145 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:19.379090    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-8qstb" podStartSLOduration=2.078936764 podStartE2EDuration="3.379074233s" totalImagesPullingTime="1.300137469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:19.377806697 +0000 UTC m=+53.840706640" watchObservedRunningTime="2026-07-07 14:27:19.379074233 +0000 UTC m=+53.841974175"
Jul 07 14:27:25.838580 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:25.838533    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:25.839135 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:25.838593    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:25.839135 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:25.838692    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 07 14:27:25.839135 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:25.838742    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls podName:fdf28b9f-131a-4f2a-94b0-a101c082c4b1 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:41.838729051 +0000 UTC m=+76.301628970 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls") pod "dns-default-95td9" (UID: "fdf28b9f-131a-4f2a-94b0-a101c082c4b1") : secret "dns-default-metrics-tls" not found
Jul 07 14:27:25.839135 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:25.838691    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 07 14:27:25.839135 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:25.838820    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls podName:fc038d77-b57f-4354-a128-672f0d0b5319 nodeName:}" failed. No retries permitted until 2026-07-07 14:27:41.838807235 +0000 UTC m=+76.301707159 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nqmhq" (UID: "fc038d77-b57f-4354-a128-672f0d0b5319") : secret "insights-runtime-extractor-tls" not found
Jul 07 14:27:30.774796 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:30.774751    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:27:30.777071 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:30.777051    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 07 14:27:30.787963 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:30.787936    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e360a2f-655d-464f-98e3-00096c75fbe9-original-pull-secret\") pod \"global-pull-secret-syncer-mk6nm\" (UID: \"7e360a2f-655d-464f-98e3-00096c75fbe9\") " pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:27:30.808101 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:30.808063    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mk6nm"
Jul 07 14:27:30.875540 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:30.875508    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs\") pod \"network-metrics-daemon-8sq9p\" (UID: \"3b53ae89-367b-4c2a-99f2-6bf60db0f958\") " pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:27:30.877966 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:30.877944    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 07 14:27:30.886062 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:30.885919    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 07 14:27:30.886062 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:27:30.886005    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs podName:3b53ae89-367b-4c2a-99f2-6bf60db0f958 nodeName:}" failed. No retries permitted until 2026-07-07 14:28:34.885969116 +0000 UTC m=+129.348869039 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs") pod "network-metrics-daemon-8sq9p" (UID: "3b53ae89-367b-4c2a-99f2-6bf60db0f958") : secret "metrics-daemon-secret" not found
Jul 07 14:27:30.948686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:30.948655    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mk6nm"]
Jul 07 14:27:30.951671 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:27:30.951642    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e360a2f_655d_464f_98e3_00096c75fbe9.slice/crio-e2097b1eb8109aaa71466d3ee792ef38835d93f29ae6159dcbf8674061930ee0 WatchSource:0}: Error finding container e2097b1eb8109aaa71466d3ee792ef38835d93f29ae6159dcbf8674061930ee0: Status 404 returned error can't find the container with id e2097b1eb8109aaa71466d3ee792ef38835d93f29ae6159dcbf8674061930ee0
Jul 07 14:27:30.976394 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:30.976364    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwvdh\" (UniqueName: \"kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh\") pod \"network-check-target-gp4zb\" (UID: \"1c687b00-93cd-4401-a4c7-04aa5c966647\") " pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:27:30.978654 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:30.978635    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 07 14:27:30.988948 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:30.988925    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 07 14:27:30.999774 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:30.999752    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwvdh\" (UniqueName: \"kubernetes.io/projected/1c687b00-93cd-4401-a4c7-04aa5c966647-kube-api-access-pwvdh\") pod \"network-check-target-gp4zb\" (UID: \"1c687b00-93cd-4401-a4c7-04aa5c966647\") " pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:27:31.115837 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:31.115755    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-29x7z\""
Jul 07 14:27:31.124399 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:31.124375    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:27:31.274227 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:31.274195    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gp4zb"]
Jul 07 14:27:31.277085 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:27:31.277054    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c687b00_93cd_4401_a4c7_04aa5c966647.slice/crio-21143b3ed60b8aab72afbc66a9debef774b3816739419c13536d1d8903cc3f5e WatchSource:0}: Error finding container 21143b3ed60b8aab72afbc66a9debef774b3816739419c13536d1d8903cc3f5e: Status 404 returned error can't find the container with id 21143b3ed60b8aab72afbc66a9debef774b3816739419c13536d1d8903cc3f5e
Jul 07 14:27:31.392230 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:31.392138    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mk6nm" event={"ID":"7e360a2f-655d-464f-98e3-00096c75fbe9","Type":"ContainerStarted","Data":"e2097b1eb8109aaa71466d3ee792ef38835d93f29ae6159dcbf8674061930ee0"}
Jul 07 14:27:31.393291 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:31.393267    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gp4zb" event={"ID":"1c687b00-93cd-4401-a4c7-04aa5c966647","Type":"ContainerStarted","Data":"21143b3ed60b8aab72afbc66a9debef774b3816739419c13536d1d8903cc3f5e"}
Jul 07 14:27:36.335764 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:36.335730    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h577t"
Jul 07 14:27:36.411731 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:36.411641    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mk6nm" event={"ID":"7e360a2f-655d-464f-98e3-00096c75fbe9","Type":"ContainerStarted","Data":"610da9fe895d735b35aa8475118108649e51830e48efc163df1c63d085e9193f"}
Jul 07 14:27:36.413050 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:36.413024    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gp4zb" event={"ID":"1c687b00-93cd-4401-a4c7-04aa5c966647","Type":"ContainerStarted","Data":"75346cfc16636eae0bf18445fa68fc4a7d3a162b9a40fb40ab10262b91eb7486"}
Jul 07 14:27:36.413173 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:36.413153    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:27:36.426779 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:36.426736    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mk6nm" podStartSLOduration=65.244007965 podStartE2EDuration="1m10.426719399s" totalImagesPullingTime="5.182711434s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:36.426064037 +0000 UTC m=+70.888963980" watchObservedRunningTime="2026-07-07 14:27:36.426719399 +0000 UTC m=+70.889619341"
Jul 07 14:27:36.441229 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:36.441188    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gp4zb" podStartSLOduration=65.587830258 podStartE2EDuration="1m10.4411721s" totalImagesPullingTime="4.853341842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:36.440444733 +0000 UTC m=+70.903344675" watchObservedRunningTime="2026-07-07 14:27:36.4411721 +0000 UTC m=+70.904072041"
Jul 07 14:27:41.343965 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.343932    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"]
Jul 07 14:27:41.346067 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.346043    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.353103 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.353083    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 07 14:27:41.353103 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.353095    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 14:27:41.353266 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.353208    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 07 14:27:41.353266 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.353208    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 14:27:41.354357 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.354344    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 07 14:27:41.354400 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.354373    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 14:27:41.354937 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.354923    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 07 14:27:41.376818 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.376789    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"]
Jul 07 14:27:41.459758 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.459728    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/702b4111-dfad-4559-ad68-27396378cf1c-hub\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.459934 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.459765    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/702b4111-dfad-4559-ad68-27396378cf1c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.459934 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.459823    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2rbh\" (UniqueName: \"kubernetes.io/projected/702b4111-dfad-4559-ad68-27396378cf1c-kube-api-access-d2rbh\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.459934 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.459842    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/702b4111-dfad-4559-ad68-27396378cf1c-ca\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.459934 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.459881    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/702b4111-dfad-4559-ad68-27396378cf1c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.459934 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.459926    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/702b4111-dfad-4559-ad68-27396378cf1c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.560283 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.560243    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2rbh\" (UniqueName: \"kubernetes.io/projected/702b4111-dfad-4559-ad68-27396378cf1c-kube-api-access-d2rbh\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.560283 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.560279    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/702b4111-dfad-4559-ad68-27396378cf1c-ca\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.560525 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.560331    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/702b4111-dfad-4559-ad68-27396378cf1c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.560525 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.560375    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/702b4111-dfad-4559-ad68-27396378cf1c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.560525 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.560411    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/702b4111-dfad-4559-ad68-27396378cf1c-hub\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.560525 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.560449    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/702b4111-dfad-4559-ad68-27396378cf1c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.565516 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.561558    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/702b4111-dfad-4559-ad68-27396378cf1c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.565516 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.563839    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/702b4111-dfad-4559-ad68-27396378cf1c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.565516 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.563873    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/702b4111-dfad-4559-ad68-27396378cf1c-hub\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.565516 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.564333    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/702b4111-dfad-4559-ad68-27396378cf1c-ca\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.566264 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.566244    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/702b4111-dfad-4559-ad68-27396378cf1c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.568348 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.568331    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2rbh\" (UniqueName: \"kubernetes.io/projected/702b4111-dfad-4559-ad68-27396378cf1c-kube-api-access-d2rbh\") pod \"cluster-proxy-proxy-agent-7d9776fcb-sqs5c\" (UID: \"702b4111-dfad-4559-ad68-27396378cf1c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.662219 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.662120    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"
Jul 07 14:27:41.787041 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.787006    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c"]
Jul 07 14:27:41.790581 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:27:41.790560    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod702b4111_dfad_4559_ad68_27396378cf1c.slice/crio-dad3372cd8158bf94d86c1e311fc3afdeb606e7b5047e8af0ab23d662db40307 WatchSource:0}: Error finding container dad3372cd8158bf94d86c1e311fc3afdeb606e7b5047e8af0ab23d662db40307: Status 404 returned error can't find the container with id dad3372cd8158bf94d86c1e311fc3afdeb606e7b5047e8af0ab23d662db40307
Jul 07 14:27:41.864609 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.864572    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:41.864784 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.864638    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:41.866872 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.866841    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdf28b9f-131a-4f2a-94b0-a101c082c4b1-metrics-tls\") pod \"dns-default-95td9\" (UID: \"fdf28b9f-131a-4f2a-94b0-a101c082c4b1\") " pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:41.866994 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:41.866908    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fc038d77-b57f-4354-a128-672f0d0b5319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nqmhq\" (UID: \"fc038d77-b57f-4354-a128-672f0d0b5319\") " pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:42.062568 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:42.062536    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fh5qh\""
Jul 07 14:27:42.071259 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:42.071235    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nqmhq"
Jul 07 14:27:42.121066 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:42.120997    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ksq8k\""
Jul 07 14:27:42.129591 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:42.129569    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:42.210677 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:42.210644    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nqmhq"]
Jul 07 14:27:42.259632 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:42.259596    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-95td9"]
Jul 07 14:27:42.262915 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:27:42.262890    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdf28b9f_131a_4f2a_94b0_a101c082c4b1.slice/crio-355a0c2ce35c1a35c86f1c98a37a04af61a2ac205d0980afd285cd8c375a61ed WatchSource:0}: Error finding container 355a0c2ce35c1a35c86f1c98a37a04af61a2ac205d0980afd285cd8c375a61ed: Status 404 returned error can't find the container with id 355a0c2ce35c1a35c86f1c98a37a04af61a2ac205d0980afd285cd8c375a61ed
Jul 07 14:27:42.431015 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:42.430971    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-95td9" event={"ID":"fdf28b9f-131a-4f2a-94b0-a101c082c4b1","Type":"ContainerStarted","Data":"355a0c2ce35c1a35c86f1c98a37a04af61a2ac205d0980afd285cd8c375a61ed"}
Jul 07 14:27:42.431961 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:42.431930    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nqmhq" event={"ID":"fc038d77-b57f-4354-a128-672f0d0b5319","Type":"ContainerStarted","Data":"b282920cf9a41c02ecc404cbcdf8a29f4a5c4e4004477c1e6eec7f045455716a"}
Jul 07 14:27:42.432959 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:42.432942    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c" event={"ID":"702b4111-dfad-4559-ad68-27396378cf1c","Type":"ContainerStarted","Data":"dad3372cd8158bf94d86c1e311fc3afdeb606e7b5047e8af0ab23d662db40307"}
Jul 07 14:27:49.461212 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:49.461174    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-95td9" event={"ID":"fdf28b9f-131a-4f2a-94b0-a101c082c4b1","Type":"ContainerStarted","Data":"1c9618793cea8ab2942791824f835a0787a5c127889eedb20e8d4bced5681e41"}
Jul 07 14:27:49.461618 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:49.461239    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-95td9" event={"ID":"fdf28b9f-131a-4f2a-94b0-a101c082c4b1","Type":"ContainerStarted","Data":"1b7a98978c7a24f7493e0e78eef0c29194ba2d9ece4beb366fdb02403bc3a1da"}
Jul 07 14:27:49.461618 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:49.461285    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-95td9"
Jul 07 14:27:49.462963 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:49.462935    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nqmhq" event={"ID":"fc038d77-b57f-4354-a128-672f0d0b5319","Type":"ContainerStarted","Data":"9fdca1b42ddc55237190b369bc50103bfc479f5ff6b29a8d64c9310b4862ea4a"}
Jul 07 14:27:49.463107 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:49.463005    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nqmhq" event={"ID":"fc038d77-b57f-4354-a128-672f0d0b5319","Type":"ContainerStarted","Data":"35e0a021a29b0486a1f826b2a19c64c2977914f8264afe30647ae53e9e9eeb74"}
Jul 07 14:27:49.464811 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:49.464792    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c" event={"ID":"702b4111-dfad-4559-ad68-27396378cf1c","Type":"ContainerStarted","Data":"0e7f46c6bf6028cb36fd20e534eea5f4dbfe66279563554c11cae8dd17a145a5"}
Jul 07 14:27:49.464901 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:49.464819    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c" event={"ID":"702b4111-dfad-4559-ad68-27396378cf1c","Type":"ContainerStarted","Data":"0a23f2382b23afecbe768e09088950b2fe56e0919a69b1baece7ecb98a27c594"}
Jul 07 14:27:49.464901 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:49.464831    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c" event={"ID":"702b4111-dfad-4559-ad68-27396378cf1c","Type":"ContainerStarted","Data":"36d9cb95e470dcd525900f3408a82cfee8b4e214070d579799b74e4555d04520"}
Jul 07 14:27:49.478289 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:49.478238    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-95td9" podStartSLOduration=34.336794084 podStartE2EDuration="40.478222065s" totalImagesPullingTime="6.141427981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:49.477662509 +0000 UTC m=+83.940562452" watchObservedRunningTime="2026-07-07 14:27:49.478222065 +0000 UTC m=+83.941122007"
Jul 07 14:27:49.496209 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:49.496149    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d9776fcb-sqs5c" podStartSLOduration=1.840117046 podStartE2EDuration="8.496132627s" totalImagesPullingTime="6.656015581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:49.495118964 +0000 UTC m=+83.958018907" watchObservedRunningTime="2026-07-07 14:27:49.496132627 +0000 UTC m=+83.959032568"
Jul 07 14:27:52.479762 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:52.479672    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nqmhq" event={"ID":"fc038d77-b57f-4354-a128-672f0d0b5319","Type":"ContainerStarted","Data":"edb467668b87737d88263b5ed98af8bfa825a1262f195d4a3b53e43f296e7888"}
Jul 07 14:27:52.499134 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:52.499082    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nqmhq" podStartSLOduration=33.621929939 podStartE2EDuration="43.499066024s" totalImagesPullingTime="9.877136085s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 14:27:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:27:52.49823499 +0000 UTC m=+86.961134933" watchObservedRunningTime="2026-07-07 14:27:52.499066024 +0000 UTC m=+86.961965944"
Jul 07 14:27:59.470898 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:27:59.470866    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-95td9"
Jul 07 14:28:07.418333 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:07.418303    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gp4zb"
Jul 07 14:28:09.322199 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.322163    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"]
Jul 07 14:28:09.353598 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.353558    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"]
Jul 07 14:28:09.353762 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.353679    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"
Jul 07 14:28:09.355724 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.355681    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 07 14:28:09.355871 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.355730    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mbgn7\""
Jul 07 14:28:09.356211 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.356194    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 07 14:28:09.356358 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.356340    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 07 14:28:09.356358 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.356314    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 07 14:28:09.356492 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.356338    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 07 14:28:09.467860 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.467817    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/456ac016-505d-4083-afce-53ae37023f6f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-5jdkl\" (UID: \"456ac016-505d-4083-afce-53ae37023f6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"
Jul 07 14:28:09.468089 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.467867    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/456ac016-505d-4083-afce-53ae37023f6f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-5jdkl\" (UID: \"456ac016-505d-4083-afce-53ae37023f6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"
Jul 07 14:28:09.468089 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.467904    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljbnk\" (UniqueName: \"kubernetes.io/projected/456ac016-505d-4083-afce-53ae37023f6f-kube-api-access-ljbnk\") pod \"prometheus-operator-567f7b8b7c-5jdkl\" (UID: \"456ac016-505d-4083-afce-53ae37023f6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"
Jul 07 14:28:09.468089 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.467939    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/456ac016-505d-4083-afce-53ae37023f6f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-5jdkl\" (UID: \"456ac016-505d-4083-afce-53ae37023f6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"
Jul 07 14:28:09.569299 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.569256    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/456ac016-505d-4083-afce-53ae37023f6f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-5jdkl\" (UID: \"456ac016-505d-4083-afce-53ae37023f6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"
Jul 07 14:28:09.569299 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.569305    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/456ac016-505d-4083-afce-53ae37023f6f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-5jdkl\" (UID: \"456ac016-505d-4083-afce-53ae37023f6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"
Jul 07 14:28:09.569565 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.569329    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljbnk\" (UniqueName: \"kubernetes.io/projected/456ac016-505d-4083-afce-53ae37023f6f-kube-api-access-ljbnk\") pod \"prometheus-operator-567f7b8b7c-5jdkl\" (UID: \"456ac016-505d-4083-afce-53ae37023f6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"
Jul 07 14:28:09.569565 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.569359    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/456ac016-505d-4083-afce-53ae37023f6f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-5jdkl\" (UID: \"456ac016-505d-4083-afce-53ae37023f6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"
Jul 07 14:28:09.569565 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:28:09.569410    2583 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 07 14:28:09.569565 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:28:09.569493    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/456ac016-505d-4083-afce-53ae37023f6f-prometheus-operator-tls podName:456ac016-505d-4083-afce-53ae37023f6f nodeName:}" failed. No retries permitted until 2026-07-07 14:28:10.069476684 +0000 UTC m=+104.532376607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/456ac016-505d-4083-afce-53ae37023f6f-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-5jdkl" (UID: "456ac016-505d-4083-afce-53ae37023f6f") : secret "prometheus-operator-tls" not found
Jul 07 14:28:09.570053 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.570036    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/456ac016-505d-4083-afce-53ae37023f6f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-5jdkl\" (UID: \"456ac016-505d-4083-afce-53ae37023f6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"
Jul 07 14:28:09.571749 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.571730    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/456ac016-505d-4083-afce-53ae37023f6f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-5jdkl\" (UID: \"456ac016-505d-4083-afce-53ae37023f6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"
Jul 07 14:28:09.577901 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:09.577837    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljbnk\" (UniqueName: \"kubernetes.io/projected/456ac016-505d-4083-afce-53ae37023f6f-kube-api-access-ljbnk\") pod \"prometheus-operator-567f7b8b7c-5jdkl\" (UID: \"456ac016-505d-4083-afce-53ae37023f6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"
Jul 07 14:28:10.073021 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:10.072993    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/456ac016-505d-4083-afce-53ae37023f6f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-5jdkl\" (UID: \"456ac016-505d-4083-afce-53ae37023f6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"
Jul 07 14:28:10.075286 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:10.075262    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/456ac016-505d-4083-afce-53ae37023f6f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-5jdkl\" (UID: \"456ac016-505d-4083-afce-53ae37023f6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"
Jul 07 14:28:10.262511 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:10.262480    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"
Jul 07 14:28:10.412824 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:10.412787    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl"]
Jul 07 14:28:10.416812 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:28:10.416771    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod456ac016_505d_4083_afce_53ae37023f6f.slice/crio-cc113ed7fcf2fc26d671a46019597814321b06f6bec5fdb3d016ef854b8abc02 WatchSource:0}: Error finding container cc113ed7fcf2fc26d671a46019597814321b06f6bec5fdb3d016ef854b8abc02: Status 404 returned error can't find the container with id cc113ed7fcf2fc26d671a46019597814321b06f6bec5fdb3d016ef854b8abc02
Jul 07 14:28:10.532861 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:10.532826    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl" event={"ID":"456ac016-505d-4083-afce-53ae37023f6f","Type":"ContainerStarted","Data":"cc113ed7fcf2fc26d671a46019597814321b06f6bec5fdb3d016ef854b8abc02"}
Jul 07 14:28:12.541394 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:12.541360    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl" event={"ID":"456ac016-505d-4083-afce-53ae37023f6f","Type":"ContainerStarted","Data":"3665caa44ff5f47c8145139b4edf0ddae2c07aea8b1d3e3e37a6ae21c208f483"}
Jul 07 14:28:12.541859 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:12.541413    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl" event={"ID":"456ac016-505d-4083-afce-53ae37023f6f","Type":"ContainerStarted","Data":"351b7323414ae5db2adc98c0c152c6c2412284f380e96b184495e88f0a6978d9"}
Jul 07 14:28:12.559600 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:12.559548    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5jdkl" podStartSLOduration=1.962885301 podStartE2EDuration="3.559530819s" totalImagesPullingTime="1.596645518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:12.558753126 +0000 UTC m=+107.021653069" watchObservedRunningTime="2026-07-07 14:28:12.559530819 +0000 UTC m=+107.022430760"
Jul 07 14:28:14.684736 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.684705    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"]
Jul 07 14:28:14.688074 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.688056    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.690108 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.690084    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 07 14:28:14.690203 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.690122    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 07 14:28:14.690683 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.690662    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 07 14:28:14.690775 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.690700    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-xmvs4\""
Jul 07 14:28:14.700409 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.700387    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"]
Jul 07 14:28:14.710943 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.710919    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kthbj"]
Jul 07 14:28:14.714124 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.714107    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.717666 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.717646    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 07 14:28:14.717802 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.717721    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 07 14:28:14.717908 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.717896    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 07 14:28:14.718139 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.718120    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nnrwj\""
Jul 07 14:28:14.811449 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.811410    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.811449 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.811448    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b58ac55e-cbd5-4601-8be8-6aca24721652-metrics-client-ca\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.811655 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.811483    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b58ac55e-cbd5-4601-8be8-6aca24721652-root\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.811655 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.811545    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b58ac55e-cbd5-4601-8be8-6aca24721652-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.811655 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.811585    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.811655 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.811608    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b58ac55e-cbd5-4601-8be8-6aca24721652-sys\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.811655 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.811637    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b58ac55e-cbd5-4601-8be8-6aca24721652-node-exporter-textfile\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.811655 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.811653    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsqxp\" (UniqueName: \"kubernetes.io/projected/b58ac55e-cbd5-4601-8be8-6aca24721652-kube-api-access-gsqxp\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.811834 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.811701    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b58ac55e-cbd5-4601-8be8-6aca24721652-node-exporter-tls\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.811834 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.811760    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.811834 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.811780    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b58ac55e-cbd5-4601-8be8-6aca24721652-node-exporter-wtmp\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.811834 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.811795    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b58ac55e-cbd5-4601-8be8-6aca24721652-node-exporter-accelerators-collector-config\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.811834 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.811819    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.812005 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.811903    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.812005 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.811920    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sxp5\" (UniqueName: \"kubernetes.io/projected/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-kube-api-access-9sxp5\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.913112 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913076    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b58ac55e-cbd5-4601-8be8-6aca24721652-node-exporter-textfile\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.913294 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913122    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsqxp\" (UniqueName: \"kubernetes.io/projected/b58ac55e-cbd5-4601-8be8-6aca24721652-kube-api-access-gsqxp\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.913294 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913145    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b58ac55e-cbd5-4601-8be8-6aca24721652-node-exporter-tls\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.913294 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913187    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.913294 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913213    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b58ac55e-cbd5-4601-8be8-6aca24721652-node-exporter-wtmp\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.913294 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913236    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b58ac55e-cbd5-4601-8be8-6aca24721652-node-exporter-accelerators-collector-config\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.913294 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913261    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.913294 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913292    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.913647 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913316    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sxp5\" (UniqueName: \"kubernetes.io/projected/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-kube-api-access-9sxp5\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.913647 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913364    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.913647 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913398    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b58ac55e-cbd5-4601-8be8-6aca24721652-metrics-client-ca\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.913647 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913437    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b58ac55e-cbd5-4601-8be8-6aca24721652-root\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.913647 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913472    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b58ac55e-cbd5-4601-8be8-6aca24721652-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.913647 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913502    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.913647 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913531    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b58ac55e-cbd5-4601-8be8-6aca24721652-sys\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.913647 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913547    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b58ac55e-cbd5-4601-8be8-6aca24721652-node-exporter-textfile\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.913647 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913616    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b58ac55e-cbd5-4601-8be8-6aca24721652-sys\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.914098 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:28:14.913680    2583 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 07 14:28:14.914098 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:28:14.913753    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-kube-state-metrics-tls podName:2bb4fee1-b8d7-4c9b-be03-55193cc5d453 nodeName:}" failed. No retries permitted until 2026-07-07 14:28:15.413731911 +0000 UTC m=+109.876631835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-5bvmm" (UID: "2bb4fee1-b8d7-4c9b-be03-55193cc5d453") : secret "kube-state-metrics-tls" not found
Jul 07 14:28:14.914098 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913763    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b58ac55e-cbd5-4601-8be8-6aca24721652-root\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.914098 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.913892    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b58ac55e-cbd5-4601-8be8-6aca24721652-node-exporter-wtmp\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.914438 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.914416    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b58ac55e-cbd5-4601-8be8-6aca24721652-node-exporter-accelerators-collector-config\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.914556 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.914521    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.914556 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.914547    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.915096 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.915073    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b58ac55e-cbd5-4601-8be8-6aca24721652-metrics-client-ca\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.915201 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.915094    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.917164 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.917139    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.917390 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.917372    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b58ac55e-cbd5-4601-8be8-6aca24721652-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.917534 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.917516    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b58ac55e-cbd5-4601-8be8-6aca24721652-node-exporter-tls\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:14.927421 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.927398    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sxp5\" (UniqueName: \"kubernetes.io/projected/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-kube-api-access-9sxp5\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:14.927541 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:14.927419    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsqxp\" (UniqueName: \"kubernetes.io/projected/b58ac55e-cbd5-4601-8be8-6aca24721652-kube-api-access-gsqxp\") pod \"node-exporter-kthbj\" (UID: \"b58ac55e-cbd5-4601-8be8-6aca24721652\") " pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:15.023620 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:15.023531    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kthbj"
Jul 07 14:28:15.035202 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:28:15.035176    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb58ac55e_cbd5_4601_8be8_6aca24721652.slice/crio-fe373aa4b1a473b53eff1c9a367a71329922c06a07591b6711640754e447be80 WatchSource:0}: Error finding container fe373aa4b1a473b53eff1c9a367a71329922c06a07591b6711640754e447be80: Status 404 returned error can't find the container with id fe373aa4b1a473b53eff1c9a367a71329922c06a07591b6711640754e447be80
Jul 07 14:28:15.417474 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:15.417394    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:15.419916 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:15.419892    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bb4fee1-b8d7-4c9b-be03-55193cc5d453-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5bvmm\" (UID: \"2bb4fee1-b8d7-4c9b-be03-55193cc5d453\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:15.552451 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:15.552414    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kthbj" event={"ID":"b58ac55e-cbd5-4601-8be8-6aca24721652","Type":"ContainerStarted","Data":"fe373aa4b1a473b53eff1c9a367a71329922c06a07591b6711640754e447be80"}
Jul 07 14:28:15.596719 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:15.596686    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"
Jul 07 14:28:15.743744 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:15.743724    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm"]
Jul 07 14:28:15.745697 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:28:15.745669    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bb4fee1_b8d7_4c9b_be03_55193cc5d453.slice/crio-eecc1236c9b065dd9ca35fe4e359e3d4ec4ed20d19222ebbf6cdb70bc7a2f529 WatchSource:0}: Error finding container eecc1236c9b065dd9ca35fe4e359e3d4ec4ed20d19222ebbf6cdb70bc7a2f529: Status 404 returned error can't find the container with id eecc1236c9b065dd9ca35fe4e359e3d4ec4ed20d19222ebbf6cdb70bc7a2f529
Jul 07 14:28:16.558422 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:16.558337    2583 generic.go:356] "Generic (PLEG): container finished" podID="b58ac55e-cbd5-4601-8be8-6aca24721652" containerID="e9cfc077ece4e38dcc7175b094642653d8479cc5fdfdc3be6a681ca1c4e04eae" exitCode=0
Jul 07 14:28:16.558591 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:16.558420    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kthbj" event={"ID":"b58ac55e-cbd5-4601-8be8-6aca24721652","Type":"ContainerDied","Data":"e9cfc077ece4e38dcc7175b094642653d8479cc5fdfdc3be6a681ca1c4e04eae"}
Jul 07 14:28:16.560030 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:16.559960    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm" event={"ID":"2bb4fee1-b8d7-4c9b-be03-55193cc5d453","Type":"ContainerStarted","Data":"eecc1236c9b065dd9ca35fe4e359e3d4ec4ed20d19222ebbf6cdb70bc7a2f529"}
Jul 07 14:28:17.567623 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:17.567521    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kthbj" event={"ID":"b58ac55e-cbd5-4601-8be8-6aca24721652","Type":"ContainerStarted","Data":"fe4e932a821296841bc6d7e1bb4667123bec627ad703f73dd0a0c16cb891b7a0"}
Jul 07 14:28:17.567623 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:17.567599    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kthbj" event={"ID":"b58ac55e-cbd5-4601-8be8-6aca24721652","Type":"ContainerStarted","Data":"f6fc77bb8cf6dc83f580da18e05cce3156e54179e4b64cc9923b072a021b35a0"}
Jul 07 14:28:17.570120 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:17.570071    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm" event={"ID":"2bb4fee1-b8d7-4c9b-be03-55193cc5d453","Type":"ContainerStarted","Data":"311113faa6f5099d8672390483fe330457322fa503652fd887e7bca3105f1c4d"}
Jul 07 14:28:17.570254 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:17.570134    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm" event={"ID":"2bb4fee1-b8d7-4c9b-be03-55193cc5d453","Type":"ContainerStarted","Data":"97a758dabbfbfd84dd61b3505e764d9eca4e60a42470b755b2168cb155e23fe6"}
Jul 07 14:28:17.588373 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:17.588319    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kthbj" podStartSLOduration=2.3231598030000002 podStartE2EDuration="3.588300536s" totalImagesPullingTime="1.199940733s" totalInitContainerRuntime="65.2ms" isStatefulPod=true podCreationTimestamp="2026-07-07 14:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:17.586805908 +0000 UTC m=+112.049705876" watchObservedRunningTime="2026-07-07 14:28:17.588300536 +0000 UTC m=+112.051200455"
Jul 07 14:28:18.575404 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:18.575367    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm" event={"ID":"2bb4fee1-b8d7-4c9b-be03-55193cc5d453","Type":"ContainerStarted","Data":"dd80e58e5d40432cb2b12bdfaf93c2c93f3dafb29aef45f4ff5bb8394497c86f"}
Jul 07 14:28:18.985233 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:18.985172    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5bvmm" podStartSLOduration=3.415827523 podStartE2EDuration="4.985145441s" totalImagesPullingTime="1.569317918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:18.595488849 +0000 UTC m=+113.058388794" watchObservedRunningTime="2026-07-07 14:28:18.985145441 +0000 UTC m=+113.448045384"
Jul 07 14:28:18.986298 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:18.986277    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-659d88c44-7n5lt"]
Jul 07 14:28:19.089809 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.089764    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-659d88c44-7n5lt"]
Jul 07 14:28:19.089969 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.089822    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.092140 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.092111    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-6fkjf\""
Jul 07 14:28:19.092271 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.092171    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 07 14:28:19.092271 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.092188    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 07 14:28:19.092271 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.092189    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 07 14:28:19.092271 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.092243    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-b5ppbrm59elao\""
Jul 07 14:28:19.092271 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.092259    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 07 14:28:19.149856 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.149817    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4b10b4b4-abd7-48a1-a9a0-821613845a27-secret-metrics-server-tls\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.149856 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.149860    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4b10b4b4-abd7-48a1-a9a0-821613845a27-metrics-server-audit-profiles\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.150138 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.149892    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b10b4b4-abd7-48a1-a9a0-821613845a27-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.150138 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.149955    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q9dm\" (UniqueName: \"kubernetes.io/projected/4b10b4b4-abd7-48a1-a9a0-821613845a27-kube-api-access-7q9dm\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.150138 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.150030    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4b10b4b4-abd7-48a1-a9a0-821613845a27-secret-metrics-server-client-certs\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.150138 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.150064    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4b10b4b4-abd7-48a1-a9a0-821613845a27-audit-log\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.150138 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.150088    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4b10b4b4-abd7-48a1-a9a0-821613845a27-client-ca-bundle\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.251062 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.250955    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4b10b4b4-abd7-48a1-a9a0-821613845a27-audit-log\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.251062 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.251015    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4b10b4b4-abd7-48a1-a9a0-821613845a27-client-ca-bundle\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.251062 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.251048    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4b10b4b4-abd7-48a1-a9a0-821613845a27-secret-metrics-server-tls\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.251315 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.251078    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4b10b4b4-abd7-48a1-a9a0-821613845a27-metrics-server-audit-profiles\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.251315 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.251102    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b10b4b4-abd7-48a1-a9a0-821613845a27-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.251315 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.251136    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q9dm\" (UniqueName: \"kubernetes.io/projected/4b10b4b4-abd7-48a1-a9a0-821613845a27-kube-api-access-7q9dm\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.251315 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.251167    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4b10b4b4-abd7-48a1-a9a0-821613845a27-secret-metrics-server-client-certs\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.251535 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.251377    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4b10b4b4-abd7-48a1-a9a0-821613845a27-audit-log\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.251826 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.251795    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b10b4b4-abd7-48a1-a9a0-821613845a27-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.252300 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.252280    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4b10b4b4-abd7-48a1-a9a0-821613845a27-metrics-server-audit-profiles\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.253735 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.253712    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4b10b4b4-abd7-48a1-a9a0-821613845a27-client-ca-bundle\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.253827 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.253708    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4b10b4b4-abd7-48a1-a9a0-821613845a27-secret-metrics-server-client-certs\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.253827 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.253752    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4b10b4b4-abd7-48a1-a9a0-821613845a27-secret-metrics-server-tls\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.259680 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.259654    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q9dm\" (UniqueName: \"kubernetes.io/projected/4b10b4b4-abd7-48a1-a9a0-821613845a27-kube-api-access-7q9dm\") pod \"metrics-server-659d88c44-7n5lt\" (UID: \"4b10b4b4-abd7-48a1-a9a0-821613845a27\") " pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.399434 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.399393    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:19.456894 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.456857    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4"]
Jul 07 14:28:19.479026 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.478968    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4"]
Jul 07 14:28:19.479191 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.479074    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4"
Jul 07 14:28:19.481260 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.481219    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 07 14:28:19.481409 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.481260    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-27hcm\""
Jul 07 14:28:19.552561 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.552517    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrp44\" (UniqueName: \"kubernetes.io/projected/b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9-kube-api-access-vrp44\") pod \"monitoring-plugin-d5dcff7cf-2jvm4\" (UID: \"b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4"
Jul 07 14:28:19.552747 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.552645    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-2jvm4\" (UID: \"b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4"
Jul 07 14:28:19.563557 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.563523    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-659d88c44-7n5lt"]
Jul 07 14:28:19.567546 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:28:19.567512    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b10b4b4_abd7_48a1_a9a0_821613845a27.slice/crio-a25dd3ee6329cf478b4aaf50caa653fc11b0490ca8e496c5457c02e30807be99 WatchSource:0}: Error finding container a25dd3ee6329cf478b4aaf50caa653fc11b0490ca8e496c5457c02e30807be99: Status 404 returned error can't find the container with id a25dd3ee6329cf478b4aaf50caa653fc11b0490ca8e496c5457c02e30807be99
Jul 07 14:28:19.579914 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.579875    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-659d88c44-7n5lt" event={"ID":"4b10b4b4-abd7-48a1-a9a0-821613845a27","Type":"ContainerStarted","Data":"a25dd3ee6329cf478b4aaf50caa653fc11b0490ca8e496c5457c02e30807be99"}
Jul 07 14:28:19.653624 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.653588    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrp44\" (UniqueName: \"kubernetes.io/projected/b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9-kube-api-access-vrp44\") pod \"monitoring-plugin-d5dcff7cf-2jvm4\" (UID: \"b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4"
Jul 07 14:28:19.653821 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.653747    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-2jvm4\" (UID: \"b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4"
Jul 07 14:28:19.653896 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:28:19.653844    2583 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 07 14:28:19.653957 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:28:19.653923    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9-monitoring-plugin-cert podName:b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9 nodeName:}" failed. No retries permitted until 2026-07-07 14:28:20.153905321 +0000 UTC m=+114.616805255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-2jvm4" (UID: "b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9") : secret "monitoring-plugin-cert" not found
Jul 07 14:28:19.662225 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:19.662193    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrp44\" (UniqueName: \"kubernetes.io/projected/b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9-kube-api-access-vrp44\") pod \"monitoring-plugin-d5dcff7cf-2jvm4\" (UID: \"b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4"
Jul 07 14:28:20.158374 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:20.158332    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-2jvm4\" (UID: \"b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4"
Jul 07 14:28:20.160683 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:20.160660    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-2jvm4\" (UID: \"b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4"
Jul 07 14:28:20.393293 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:20.393249    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4"
Jul 07 14:28:20.578817 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:20.578783    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4"]
Jul 07 14:28:20.588119 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:28:20.588087    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1c71ac0_9506_4612_9bf0_d2f5ce17b1b9.slice/crio-f10688c0fd58c0a2c4752ab8ac6b02b78d811f353223a192b5df5a2e0cee3c3c WatchSource:0}: Error finding container f10688c0fd58c0a2c4752ab8ac6b02b78d811f353223a192b5df5a2e0cee3c3c: Status 404 returned error can't find the container with id f10688c0fd58c0a2c4752ab8ac6b02b78d811f353223a192b5df5a2e0cee3c3c
Jul 07 14:28:21.588374 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:21.588344    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4" event={"ID":"b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9","Type":"ContainerStarted","Data":"f10688c0fd58c0a2c4752ab8ac6b02b78d811f353223a192b5df5a2e0cee3c3c"}
Jul 07 14:28:22.593470 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:22.593430    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-659d88c44-7n5lt" event={"ID":"4b10b4b4-abd7-48a1-a9a0-821613845a27","Type":"ContainerStarted","Data":"6bcf43ea85b06e10c3090bf8235b00c4b205821b7a9a3478df4863a8831ede94"}
Jul 07 14:28:22.612657 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:22.612606    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-659d88c44-7n5lt" podStartSLOduration=2.353290969 podStartE2EDuration="4.61258736s" totalImagesPullingTime="2.259296391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:28:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:22.611673061 +0000 UTC m=+117.074573005" watchObservedRunningTime="2026-07-07 14:28:22.61258736 +0000 UTC m=+117.075487301"
Jul 07 14:28:23.598216 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:23.598183    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4" event={"ID":"b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9","Type":"ContainerStarted","Data":"e5d29a5c471f296afa77f26d6bbad7f457140056d3179dd14e68ca60adab6c5c"}
Jul 07 14:28:23.598689 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:23.598443    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4"
Jul 07 14:28:23.603204 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:23.603182    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4"
Jul 07 14:28:23.618234 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:23.618184    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2jvm4" podStartSLOduration=2.165622014 podStartE2EDuration="4.618163716s" totalImagesPullingTime="2.452541702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:28:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:23.616159794 +0000 UTC m=+118.079059735" watchObservedRunningTime="2026-07-07 14:28:23.618163716 +0000 UTC m=+118.081063662"
Jul 07 14:28:24.907165 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:24.907136    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5bvmm_2bb4fee1-b8d7-4c9b-be03-55193cc5d453/kube-state-metrics/0.log"
Jul 07 14:28:25.106913 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:25.106886    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5bvmm_2bb4fee1-b8d7-4c9b-be03-55193cc5d453/kube-rbac-proxy-main/0.log"
Jul 07 14:28:25.306471 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:25.306439    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5bvmm_2bb4fee1-b8d7-4c9b-be03-55193cc5d453/kube-rbac-proxy-self/0.log"
Jul 07 14:28:25.506838 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:25.506813    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-659d88c44-7n5lt_4b10b4b4-abd7-48a1-a9a0-821613845a27/metrics-server/0.log"
Jul 07 14:28:25.706855 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:25.706832    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-2jvm4_b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9/monitoring-plugin/0.log"
Jul 07 14:28:26.508901 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:26.508874    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kthbj_b58ac55e-cbd5-4601-8be8-6aca24721652/init-textfile/0.log"
Jul 07 14:28:26.711395 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:26.711372    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kthbj_b58ac55e-cbd5-4601-8be8-6aca24721652/node-exporter/0.log"
Jul 07 14:28:26.910049 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:26.909954    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kthbj_b58ac55e-cbd5-4601-8be8-6aca24721652/kube-rbac-proxy/0.log"
Jul 07 14:28:29.707906 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:29.707874    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-5jdkl_456ac016-505d-4083-afce-53ae37023f6f/prometheus-operator/0.log"
Jul 07 14:28:29.906992 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:29.906951    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-5jdkl_456ac016-505d-4083-afce-53ae37023f6f/kube-rbac-proxy/0.log"
Jul 07 14:28:33.106747 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:33.106715    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9tpzt_160fe77b-4e72-4855-9eb0-ac0b2708177c/serve-healthcheck-canary/0.log"
Jul 07 14:28:34.970571 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:34.970523    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs\") pod \"network-metrics-daemon-8sq9p\" (UID: \"3b53ae89-367b-4c2a-99f2-6bf60db0f958\") " pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:28:34.973877 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:34.973835    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b53ae89-367b-4c2a-99f2-6bf60db0f958-metrics-certs\") pod \"network-metrics-daemon-8sq9p\" (UID: \"3b53ae89-367b-4c2a-99f2-6bf60db0f958\") " pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:28:35.023931 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:35.023663    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ps6h4\""
Jul 07 14:28:35.029625 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:35.029598    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8sq9p"
Jul 07 14:28:35.200765 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:35.200729    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8sq9p"]
Jul 07 14:28:35.203538 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:28:35.203511    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b53ae89_367b_4c2a_99f2_6bf60db0f958.slice/crio-596735fecb9d161c97421f264738197288fffadb38f4ff3cb53186bc4fb3d252 WatchSource:0}: Error finding container 596735fecb9d161c97421f264738197288fffadb38f4ff3cb53186bc4fb3d252: Status 404 returned error can't find the container with id 596735fecb9d161c97421f264738197288fffadb38f4ff3cb53186bc4fb3d252
Jul 07 14:28:35.640926 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:35.640851    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8sq9p" event={"ID":"3b53ae89-367b-4c2a-99f2-6bf60db0f958","Type":"ContainerStarted","Data":"596735fecb9d161c97421f264738197288fffadb38f4ff3cb53186bc4fb3d252"}
Jul 07 14:28:37.650132 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:37.650094    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8sq9p" event={"ID":"3b53ae89-367b-4c2a-99f2-6bf60db0f958","Type":"ContainerStarted","Data":"d4448f03bda9544746a3e46377a235bf2ce9b3ecaec9a2b8618ba12e508b71e9"}
Jul 07 14:28:37.650603 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:37.650196    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8sq9p" event={"ID":"3b53ae89-367b-4c2a-99f2-6bf60db0f958","Type":"ContainerStarted","Data":"80bd4f0c46fd702fe2c4402bb2e8726098e8be8c429718429d121f15ebea299a"}
Jul 07 14:28:37.666388 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:37.666331    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8sq9p" podStartSLOduration=130.25252276 podStartE2EDuration="2m11.666309026s" totalImagesPullingTime="1.413786266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:28:37.665752903 +0000 UTC m=+132.128652847" watchObservedRunningTime="2026-07-07 14:28:37.666309026 +0000 UTC m=+132.129208968"
Jul 07 14:28:39.400062 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:39.400024    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:39.400458 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:39.400092    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:59.405667 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:59.405636    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:28:59.409684 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:28:59.409657    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-659d88c44-7n5lt"
Jul 07 14:31:25.963436 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:31:25.963405    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:31:25.970546 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:31:25.970521    2583 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 14:31:25.970864 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:31:25.970848    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:32:15.705109 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:15.705066    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-drb4t"]
Jul 07 14:32:15.708333 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:15.708310    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-drb4t"
Jul 07 14:32:15.710613 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:15.710587    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 07 14:32:15.710923 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:15.710906    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-ddp5q\""
Jul 07 14:32:15.711002 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:15.710915    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 07 14:32:15.719710 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:15.719683    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-drb4t"]
Jul 07 14:32:15.793969 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:15.793928    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9bba008-818f-4214-b502-067d67a88004-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-drb4t\" (UID: \"a9bba008-818f-4214-b502-067d67a88004\") " pod="cert-manager/cert-manager-webhook-6486ffd997-drb4t"
Jul 07 14:32:15.793969 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:15.794004    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk6qb\" (UniqueName: \"kubernetes.io/projected/a9bba008-818f-4214-b502-067d67a88004-kube-api-access-zk6qb\") pod \"cert-manager-webhook-6486ffd997-drb4t\" (UID: \"a9bba008-818f-4214-b502-067d67a88004\") " pod="cert-manager/cert-manager-webhook-6486ffd997-drb4t"
Jul 07 14:32:15.895120 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:15.895080    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9bba008-818f-4214-b502-067d67a88004-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-drb4t\" (UID: \"a9bba008-818f-4214-b502-067d67a88004\") " pod="cert-manager/cert-manager-webhook-6486ffd997-drb4t"
Jul 07 14:32:15.895264 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:15.895129    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk6qb\" (UniqueName: \"kubernetes.io/projected/a9bba008-818f-4214-b502-067d67a88004-kube-api-access-zk6qb\") pod \"cert-manager-webhook-6486ffd997-drb4t\" (UID: \"a9bba008-818f-4214-b502-067d67a88004\") " pod="cert-manager/cert-manager-webhook-6486ffd997-drb4t"
Jul 07 14:32:15.903854 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:15.903821    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9bba008-818f-4214-b502-067d67a88004-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-drb4t\" (UID: \"a9bba008-818f-4214-b502-067d67a88004\") " pod="cert-manager/cert-manager-webhook-6486ffd997-drb4t"
Jul 07 14:32:15.903854 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:15.903849    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk6qb\" (UniqueName: \"kubernetes.io/projected/a9bba008-818f-4214-b502-067d67a88004-kube-api-access-zk6qb\") pod \"cert-manager-webhook-6486ffd997-drb4t\" (UID: \"a9bba008-818f-4214-b502-067d67a88004\") " pod="cert-manager/cert-manager-webhook-6486ffd997-drb4t"
Jul 07 14:32:16.018932 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:16.018842    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-drb4t"
Jul 07 14:32:16.145853 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:16.145823    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-drb4t"]
Jul 07 14:32:16.148816 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:32:16.148783    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9bba008_818f_4214_b502_067d67a88004.slice/crio-28511340cf800bd94043972a84b42180533c083f3583b3349c0194bbadaeb4b1 WatchSource:0}: Error finding container 28511340cf800bd94043972a84b42180533c083f3583b3349c0194bbadaeb4b1: Status 404 returned error can't find the container with id 28511340cf800bd94043972a84b42180533c083f3583b3349c0194bbadaeb4b1
Jul 07 14:32:16.150698 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:16.150680    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 14:32:16.409008 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:16.408902    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-drb4t" event={"ID":"a9bba008-818f-4214-b502-067d67a88004","Type":"ContainerStarted","Data":"28511340cf800bd94043972a84b42180533c083f3583b3349c0194bbadaeb4b1"}
Jul 07 14:32:19.422678 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:19.422587    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-drb4t" event={"ID":"a9bba008-818f-4214-b502-067d67a88004","Type":"ContainerStarted","Data":"fafb82c3e5cc78d26d61d4a0e12d96ea372883d7f4c7d770836f8f97609610cb"}
Jul 07 14:32:19.423129 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:19.422704    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-drb4t"
Jul 07 14:32:19.439712 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:19.439656    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-drb4t" podStartSLOduration=1.542128988 podStartE2EDuration="4.439637553s" totalImagesPullingTime="2.897508565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:32:19.438388759 +0000 UTC m=+353.901288702" watchObservedRunningTime="2026-07-07 14:32:19.439637553 +0000 UTC m=+353.902537493"
Jul 07 14:32:25.427553 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:25.427517    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-drb4t"
Jul 07 14:32:43.557458 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.557425    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"]
Jul 07 14:32:43.562806 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.562788    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.574009 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.570935    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 14:32:43.575355 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.575324    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 14:32:43.575729 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.575707    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 14:32:43.575729 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.575724    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-g2dhx\""
Jul 07 14:32:43.575894 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.575750    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 14:32:43.597190 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.597159    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"]
Jul 07 14:32:43.604467 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.604439    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9673ca4b-abb2-4005-897f-e7fa928de2ab-tmp\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-6spbq\" (UID: \"9673ca4b-abb2-4005-897f-e7fa928de2ab\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.604630 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.604476    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrn74\" (UniqueName: \"kubernetes.io/projected/9673ca4b-abb2-4005-897f-e7fa928de2ab-kube-api-access-hrn74\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-6spbq\" (UID: \"9673ca4b-abb2-4005-897f-e7fa928de2ab\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.604630 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.604512    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9673ca4b-abb2-4005-897f-e7fa928de2ab-manifests\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-6spbq\" (UID: \"9673ca4b-abb2-4005-897f-e7fa928de2ab\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.604630 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.604570    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9673ca4b-abb2-4005-897f-e7fa928de2ab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-6spbq\" (UID: \"9673ca4b-abb2-4005-897f-e7fa928de2ab\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.604630 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.604586    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9673ca4b-abb2-4005-897f-e7fa928de2ab-webhook-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-6spbq\" (UID: \"9673ca4b-abb2-4005-897f-e7fa928de2ab\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.705832 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.705799    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9673ca4b-abb2-4005-897f-e7fa928de2ab-tmp\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-6spbq\" (UID: \"9673ca4b-abb2-4005-897f-e7fa928de2ab\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.705832 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.705840    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrn74\" (UniqueName: \"kubernetes.io/projected/9673ca4b-abb2-4005-897f-e7fa928de2ab-kube-api-access-hrn74\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-6spbq\" (UID: \"9673ca4b-abb2-4005-897f-e7fa928de2ab\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.706127 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.706002    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9673ca4b-abb2-4005-897f-e7fa928de2ab-manifests\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-6spbq\" (UID: \"9673ca4b-abb2-4005-897f-e7fa928de2ab\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.706127 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.706063    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9673ca4b-abb2-4005-897f-e7fa928de2ab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-6spbq\" (UID: \"9673ca4b-abb2-4005-897f-e7fa928de2ab\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.706127 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.706090    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9673ca4b-abb2-4005-897f-e7fa928de2ab-webhook-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-6spbq\" (UID: \"9673ca4b-abb2-4005-897f-e7fa928de2ab\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.706507 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.706472    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9673ca4b-abb2-4005-897f-e7fa928de2ab-manifests\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-6spbq\" (UID: \"9673ca4b-abb2-4005-897f-e7fa928de2ab\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.708296 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.708273    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9673ca4b-abb2-4005-897f-e7fa928de2ab-tmp\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-6spbq\" (UID: \"9673ca4b-abb2-4005-897f-e7fa928de2ab\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.708407 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.708380    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9673ca4b-abb2-4005-897f-e7fa928de2ab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-6spbq\" (UID: \"9673ca4b-abb2-4005-897f-e7fa928de2ab\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.708483 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.708464    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9673ca4b-abb2-4005-897f-e7fa928de2ab-webhook-cert\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-6spbq\" (UID: \"9673ca4b-abb2-4005-897f-e7fa928de2ab\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.716992 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.716944    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrn74\" (UniqueName: \"kubernetes.io/projected/9673ca4b-abb2-4005-897f-e7fa928de2ab-kube-api-access-hrn74\") pod \"opendatahub-operator-controller-manager-7868f6cbb6-6spbq\" (UID: \"9673ca4b-abb2-4005-897f-e7fa928de2ab\") " pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:43.876097 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:43.875995    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:44.034034 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:44.033999    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"]
Jul 07 14:32:44.037911 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:32:44.037888    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9673ca4b_abb2_4005_897f_e7fa928de2ab.slice/crio-4d727d2f951900097e73882e4d458cf42f88ac89990a3e0369af134e60e811c5 WatchSource:0}: Error finding container 4d727d2f951900097e73882e4d458cf42f88ac89990a3e0369af134e60e811c5: Status 404 returned error can't find the container with id 4d727d2f951900097e73882e4d458cf42f88ac89990a3e0369af134e60e811c5
Jul 07 14:32:44.512665 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:44.512626    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq" event={"ID":"9673ca4b-abb2-4005-897f-e7fa928de2ab","Type":"ContainerStarted","Data":"4d727d2f951900097e73882e4d458cf42f88ac89990a3e0369af134e60e811c5"}
Jul 07 14:32:48.533874 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:48.533839    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq" event={"ID":"9673ca4b-abb2-4005-897f-e7fa928de2ab","Type":"ContainerStarted","Data":"0c9efb85940b4aa38575d31570a960366f6b335261a4c51bab99fe20daa32575"}
Jul 07 14:32:49.538664 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:49.538631    2583 generic.go:356] "Generic (PLEG): container finished" podID="9673ca4b-abb2-4005-897f-e7fa928de2ab" containerID="0c9efb85940b4aa38575d31570a960366f6b335261a4c51bab99fe20daa32575" exitCode=0
Jul 07 14:32:49.539132 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:49.538670    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq" event={"ID":"9673ca4b-abb2-4005-897f-e7fa928de2ab","Type":"ContainerDied","Data":"0c9efb85940b4aa38575d31570a960366f6b335261a4c51bab99fe20daa32575"}
Jul 07 14:32:51.547179 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:51.547141    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq" event={"ID":"9673ca4b-abb2-4005-897f-e7fa928de2ab","Type":"ContainerStarted","Data":"3c9290d94c6ea172751cd7e6beafb231e8d07ec93f1eb55311d55bda2e545fe5"}
Jul 07 14:32:51.547589 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:51.547358    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:32:51.570182 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:32:51.570123    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq" podStartSLOduration=2.359358401 podStartE2EDuration="8.570100789s" totalImagesPullingTime="5.121552445s" totalInitContainerRuntime="1.089189943s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:32:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:32:51.567858945 +0000 UTC m=+386.030758888" watchObservedRunningTime="2026-07-07 14:32:51.570100789 +0000 UTC m=+386.033000731"
Jul 07 14:33:02.552542 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:02.552513    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7868f6cbb6-6spbq"
Jul 07 14:33:05.062619 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.062583    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc"]
Jul 07 14:33:05.070674 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.070646    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc"
Jul 07 14:33:05.073696 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.073661    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 14:33:05.073696 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.073662    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 14:33:05.073897 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.073700    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 07 14:33:05.073897 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.073662    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 07 14:33:05.074292 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.074275    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mz56n\""
Jul 07 14:33:05.085127 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.085101    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc"]
Jul 07 14:33:05.175373 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.175339    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fq86n"]
Jul 07 14:33:05.178529 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.178510    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fq86n"
Jul 07 14:33:05.181381 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.181341    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-89mjs\""
Jul 07 14:33:05.181544 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.181458    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 07 14:33:05.187787 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.187764    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/738fc825-7f18-4e35-b44e-b97bd47f9bf6-tls-certs\") pod \"kube-auth-proxy-7646845f6c-cdhzc\" (UID: \"738fc825-7f18-4e35-b44e-b97bd47f9bf6\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc"
Jul 07 14:33:05.187889 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.187808    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/738fc825-7f18-4e35-b44e-b97bd47f9bf6-tmp\") pod \"kube-auth-proxy-7646845f6c-cdhzc\" (UID: \"738fc825-7f18-4e35-b44e-b97bd47f9bf6\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc"
Jul 07 14:33:05.187889 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.187826    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b527d\" (UniqueName: \"kubernetes.io/projected/738fc825-7f18-4e35-b44e-b97bd47f9bf6-kube-api-access-b527d\") pod \"kube-auth-proxy-7646845f6c-cdhzc\" (UID: \"738fc825-7f18-4e35-b44e-b97bd47f9bf6\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc"
Jul 07 14:33:05.193794 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.193762    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fq86n"]
Jul 07 14:33:05.288688 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.288654    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6597e7e8-e649-4d74-8d04-15c8fad9b423-cert\") pod \"odh-model-controller-7998c67dc4-fq86n\" (UID: \"6597e7e8-e649-4d74-8d04-15c8fad9b423\") " pod="opendatahub/odh-model-controller-7998c67dc4-fq86n"
Jul 07 14:33:05.288904 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.288697    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/738fc825-7f18-4e35-b44e-b97bd47f9bf6-tls-certs\") pod \"kube-auth-proxy-7646845f6c-cdhzc\" (UID: \"738fc825-7f18-4e35-b44e-b97bd47f9bf6\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc"
Jul 07 14:33:05.288904 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.288749    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r49fd\" (UniqueName: \"kubernetes.io/projected/6597e7e8-e649-4d74-8d04-15c8fad9b423-kube-api-access-r49fd\") pod \"odh-model-controller-7998c67dc4-fq86n\" (UID: \"6597e7e8-e649-4d74-8d04-15c8fad9b423\") " pod="opendatahub/odh-model-controller-7998c67dc4-fq86n"
Jul 07 14:33:05.288904 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.288775    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/738fc825-7f18-4e35-b44e-b97bd47f9bf6-tmp\") pod \"kube-auth-proxy-7646845f6c-cdhzc\" (UID: \"738fc825-7f18-4e35-b44e-b97bd47f9bf6\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc"
Jul 07 14:33:05.288904 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.288791    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b527d\" (UniqueName: \"kubernetes.io/projected/738fc825-7f18-4e35-b44e-b97bd47f9bf6-kube-api-access-b527d\") pod \"kube-auth-proxy-7646845f6c-cdhzc\" (UID: \"738fc825-7f18-4e35-b44e-b97bd47f9bf6\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc"
Jul 07 14:33:05.291207 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.291173    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/738fc825-7f18-4e35-b44e-b97bd47f9bf6-tmp\") pod \"kube-auth-proxy-7646845f6c-cdhzc\" (UID: \"738fc825-7f18-4e35-b44e-b97bd47f9bf6\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc"
Jul 07 14:33:05.291396 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.291378    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/738fc825-7f18-4e35-b44e-b97bd47f9bf6-tls-certs\") pod \"kube-auth-proxy-7646845f6c-cdhzc\" (UID: \"738fc825-7f18-4e35-b44e-b97bd47f9bf6\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc"
Jul 07 14:33:05.300415 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.300385    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b527d\" (UniqueName: \"kubernetes.io/projected/738fc825-7f18-4e35-b44e-b97bd47f9bf6-kube-api-access-b527d\") pod \"kube-auth-proxy-7646845f6c-cdhzc\" (UID: \"738fc825-7f18-4e35-b44e-b97bd47f9bf6\") " pod="openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc"
Jul 07 14:33:05.380575 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.380480    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc"
Jul 07 14:33:05.389991 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.389949    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6597e7e8-e649-4d74-8d04-15c8fad9b423-cert\") pod \"odh-model-controller-7998c67dc4-fq86n\" (UID: \"6597e7e8-e649-4d74-8d04-15c8fad9b423\") " pod="opendatahub/odh-model-controller-7998c67dc4-fq86n"
Jul 07 14:33:05.390125 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.390035    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r49fd\" (UniqueName: \"kubernetes.io/projected/6597e7e8-e649-4d74-8d04-15c8fad9b423-kube-api-access-r49fd\") pod \"odh-model-controller-7998c67dc4-fq86n\" (UID: \"6597e7e8-e649-4d74-8d04-15c8fad9b423\") " pod="opendatahub/odh-model-controller-7998c67dc4-fq86n"
Jul 07 14:33:05.392596 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.392566    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6597e7e8-e649-4d74-8d04-15c8fad9b423-cert\") pod \"odh-model-controller-7998c67dc4-fq86n\" (UID: \"6597e7e8-e649-4d74-8d04-15c8fad9b423\") " pod="opendatahub/odh-model-controller-7998c67dc4-fq86n"
Jul 07 14:33:05.399943 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.399904    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r49fd\" (UniqueName: \"kubernetes.io/projected/6597e7e8-e649-4d74-8d04-15c8fad9b423-kube-api-access-r49fd\") pod \"odh-model-controller-7998c67dc4-fq86n\" (UID: \"6597e7e8-e649-4d74-8d04-15c8fad9b423\") " pod="opendatahub/odh-model-controller-7998c67dc4-fq86n"
Jul 07 14:33:05.488377 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.488343    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-fq86n"
Jul 07 14:33:05.548618 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.548590    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc"]
Jul 07 14:33:05.551620 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:33:05.551552    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod738fc825_7f18_4e35_b44e_b97bd47f9bf6.slice/crio-1abfca6b21bb21de51641fefe98981c00c2bb3195ad6a173054269bc87766e91 WatchSource:0}: Error finding container 1abfca6b21bb21de51641fefe98981c00c2bb3195ad6a173054269bc87766e91: Status 404 returned error can't find the container with id 1abfca6b21bb21de51641fefe98981c00c2bb3195ad6a173054269bc87766e91
Jul 07 14:33:05.599765 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.599715    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc" event={"ID":"738fc825-7f18-4e35-b44e-b97bd47f9bf6","Type":"ContainerStarted","Data":"1abfca6b21bb21de51641fefe98981c00c2bb3195ad6a173054269bc87766e91"}
Jul 07 14:33:05.648495 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:05.648466    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-fq86n"]
Jul 07 14:33:05.650741 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:33:05.650707    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6597e7e8_e649_4d74_8d04_15c8fad9b423.slice/crio-b464efd19015116c509e34f634f09abcf380225fb8f80fa6fae6941c67854837 WatchSource:0}: Error finding container b464efd19015116c509e34f634f09abcf380225fb8f80fa6fae6941c67854837: Status 404 returned error can't find the container with id b464efd19015116c509e34f634f09abcf380225fb8f80fa6fae6941c67854837
Jul 07 14:33:06.150800 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.150763    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cm5cr"]
Jul 07 14:33:06.154160 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.154133    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cm5cr"
Jul 07 14:33:06.157123 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.157100    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-vblvg\""
Jul 07 14:33:06.157522 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.157500    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 07 14:33:06.172566 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.172519    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cm5cr"]
Jul 07 14:33:06.296218 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.296172    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae372a71-bd3b-4cd6-a899-ecc7e3740d8d-tls-certs\") pod \"model-serving-api-54d89bd5fc-cm5cr\" (UID: \"ae372a71-bd3b-4cd6-a899-ecc7e3740d8d\") " pod="opendatahub/model-serving-api-54d89bd5fc-cm5cr"
Jul 07 14:33:06.296381 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.296321    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc5zq\" (UniqueName: \"kubernetes.io/projected/ae372a71-bd3b-4cd6-a899-ecc7e3740d8d-kube-api-access-mc5zq\") pod \"model-serving-api-54d89bd5fc-cm5cr\" (UID: \"ae372a71-bd3b-4cd6-a899-ecc7e3740d8d\") " pod="opendatahub/model-serving-api-54d89bd5fc-cm5cr"
Jul 07 14:33:06.397284 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.397250    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc5zq\" (UniqueName: \"kubernetes.io/projected/ae372a71-bd3b-4cd6-a899-ecc7e3740d8d-kube-api-access-mc5zq\") pod \"model-serving-api-54d89bd5fc-cm5cr\" (UID: \"ae372a71-bd3b-4cd6-a899-ecc7e3740d8d\") " pod="opendatahub/model-serving-api-54d89bd5fc-cm5cr"
Jul 07 14:33:06.397471 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.397322    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae372a71-bd3b-4cd6-a899-ecc7e3740d8d-tls-certs\") pod \"model-serving-api-54d89bd5fc-cm5cr\" (UID: \"ae372a71-bd3b-4cd6-a899-ecc7e3740d8d\") " pod="opendatahub/model-serving-api-54d89bd5fc-cm5cr"
Jul 07 14:33:06.400216 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.400184    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae372a71-bd3b-4cd6-a899-ecc7e3740d8d-tls-certs\") pod \"model-serving-api-54d89bd5fc-cm5cr\" (UID: \"ae372a71-bd3b-4cd6-a899-ecc7e3740d8d\") " pod="opendatahub/model-serving-api-54d89bd5fc-cm5cr"
Jul 07 14:33:06.405880 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.405769    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc5zq\" (UniqueName: \"kubernetes.io/projected/ae372a71-bd3b-4cd6-a899-ecc7e3740d8d-kube-api-access-mc5zq\") pod \"model-serving-api-54d89bd5fc-cm5cr\" (UID: \"ae372a71-bd3b-4cd6-a899-ecc7e3740d8d\") " pod="opendatahub/model-serving-api-54d89bd5fc-cm5cr"
Jul 07 14:33:06.466748 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.466712    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cm5cr"
Jul 07 14:33:06.606346 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.606312    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fq86n" event={"ID":"6597e7e8-e649-4d74-8d04-15c8fad9b423","Type":"ContainerStarted","Data":"b464efd19015116c509e34f634f09abcf380225fb8f80fa6fae6941c67854837"}
Jul 07 14:33:06.615670 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.615639    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"]
Jul 07 14:33:06.623333 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.623304    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.625511 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.625460    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 07 14:33:06.625511 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.625502    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 07 14:33:06.625700 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.625477    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 07 14:33:06.625700 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.625633    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-m8nrh\""
Jul 07 14:33:06.626407 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.626392    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 14:33:06.630728 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.630704    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cm5cr"]
Jul 07 14:33:06.634488 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.634460    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"]
Jul 07 14:33:06.634590 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:33:06.634509    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae372a71_bd3b_4cd6_a899_ecc7e3740d8d.slice/crio-ed403bee55f3caeaeec6d7927944f37a30ff07ea603d354ccbb952d43f8b2035 WatchSource:0}: Error finding container ed403bee55f3caeaeec6d7927944f37a30ff07ea603d354ccbb952d43f8b2035: Status 404 returned error can't find the container with id ed403bee55f3caeaeec6d7927944f37a30ff07ea603d354ccbb952d43f8b2035
Jul 07 14:33:06.700415 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.700379    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk5q9\" (UniqueName: \"kubernetes.io/projected/f5036107-8b95-4f86-a966-e14b6a5bd522-kube-api-access-bk5q9\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.700603 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.700423    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.700603 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.700454    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-cacerts\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.700603 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.700567    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.700603 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.700599    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.700802 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.700696    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-token\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.700802 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.700743    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5036107-8b95-4f86-a966-e14b6a5bd522-local-certs\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.801727 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.801682    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.801727 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.801729    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.802003 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.801773    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-token\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.802003 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.801801    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5036107-8b95-4f86-a966-e14b6a5bd522-local-certs\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.802003 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.801828    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk5q9\" (UniqueName: \"kubernetes.io/projected/f5036107-8b95-4f86-a966-e14b6a5bd522-kube-api-access-bk5q9\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.802003 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.801854    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.802003 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.801897    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-cacerts\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.802600 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.802570    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.804261 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.804231    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.804535 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.804511    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5036107-8b95-4f86-a966-e14b6a5bd522-local-certs\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.804676 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.804656    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.804765 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.804744    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-cacerts\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.810026 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.810000    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-token\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.810251 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.810236    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk5q9\" (UniqueName: \"kubernetes.io/projected/f5036107-8b95-4f86-a966-e14b6a5bd522-kube-api-access-bk5q9\") pod \"istiod-openshift-gateway-7574459656-2bvxm\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:06.934171 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:06.934120    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:07.079617 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:07.079555    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"]
Jul 07 14:33:07.080846 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:33:07.080774    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5036107_8b95_4f86_a966_e14b6a5bd522.slice/crio-616d997c339d37b8d6da0a7efb7dba39a10606c989edd5724724253ae2f85704 WatchSource:0}: Error finding container 616d997c339d37b8d6da0a7efb7dba39a10606c989edd5724724253ae2f85704: Status 404 returned error can't find the container with id 616d997c339d37b8d6da0a7efb7dba39a10606c989edd5724724253ae2f85704
Jul 07 14:33:07.612372 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:07.612314    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cm5cr" event={"ID":"ae372a71-bd3b-4cd6-a899-ecc7e3740d8d","Type":"ContainerStarted","Data":"ed403bee55f3caeaeec6d7927944f37a30ff07ea603d354ccbb952d43f8b2035"}
Jul 07 14:33:07.613812 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:07.613771    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm" event={"ID":"f5036107-8b95-4f86-a966-e14b6a5bd522","Type":"ContainerStarted","Data":"616d997c339d37b8d6da0a7efb7dba39a10606c989edd5724724253ae2f85704"}
Jul 07 14:33:11.344729 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:11.344681    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 14:33:11.345054 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:11.344773    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 07 14:33:11.638215 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:11.638107    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc" event={"ID":"738fc825-7f18-4e35-b44e-b97bd47f9bf6","Type":"ContainerStarted","Data":"f4b39d109780fc8426eaf4567b8df22cb8759de457510cb9c6e9735464427d27"}
Jul 07 14:33:11.639966 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:11.639932    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cm5cr" event={"ID":"ae372a71-bd3b-4cd6-a899-ecc7e3740d8d","Type":"ContainerStarted","Data":"ecb1c3159dacedbf768dbd1c7b83297a2c7e44e1bb122f189a233740da6a7644"}
Jul 07 14:33:11.640164 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:11.640031    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-cm5cr"
Jul 07 14:33:11.641716 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:11.641690    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-fq86n" event={"ID":"6597e7e8-e649-4d74-8d04-15c8fad9b423","Type":"ContainerStarted","Data":"ce0289192d03ff34d2775bc8ac6f747886735f7944e875652eb671ef714b5738"}
Jul 07 14:33:11.641846 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:11.641786    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-fq86n"
Jul 07 14:33:11.643718 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:11.643693    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm" event={"ID":"f5036107-8b95-4f86-a966-e14b6a5bd522","Type":"ContainerStarted","Data":"008552afd4f6d836a8cb60703d39a76f5463156957f3f005dc7b84ee73ca2201"}
Jul 07 14:33:11.644011 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:11.643950    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:11.689174 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:11.688548    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7646845f6c-cdhzc" podStartSLOduration=0.961959201 podStartE2EDuration="6.688525216s" totalImagesPullingTime="5.726566015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:11.661667769 +0000 UTC m=+406.124567711" watchObservedRunningTime="2026-07-07 14:33:11.688525216 +0000 UTC m=+406.151425160"
Jul 07 14:33:11.689674 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:11.689576    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-cm5cr" podStartSLOduration=1.045650486 podStartE2EDuration="5.689557532s" totalImagesPullingTime="4.643907046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:11.688472413 +0000 UTC m=+406.151372356" watchObservedRunningTime="2026-07-07 14:33:11.689557532 +0000 UTC m=+406.152457476"
Jul 07 14:33:11.714085 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:11.713620    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-fq86n" podStartSLOduration=1.085112904 podStartE2EDuration="6.713597767s" totalImagesPullingTime="5.628484863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:11.713422097 +0000 UTC m=+406.176322039" watchObservedRunningTime="2026-07-07 14:33:11.713597767 +0000 UTC m=+406.176497722"
Jul 07 14:33:11.736175 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:11.736125    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm" podStartSLOduration=1.475107464 podStartE2EDuration="5.736106849s" totalImagesPullingTime="4.260999385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:11.734623346 +0000 UTC m=+406.197523289" watchObservedRunningTime="2026-07-07 14:33:11.736106849 +0000 UTC m=+406.199006789"
Jul 07 14:33:12.649748 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:12.649702    2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-2bvxm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 14:33:12.650166 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:12.649775    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm" podUID="f5036107-8b95-4f86-a966-e14b6a5bd522" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 14:33:14.081521 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.081492    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"]
Jul 07 14:33:14.084722 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.084706    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:33:14.087489 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.087463    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 07 14:33:14.087615 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.087571    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 07 14:33:14.088041 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.088022    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 07 14:33:14.089610 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.089590    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lxhqm\""
Jul 07 14:33:14.091892 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.091875    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 07 14:33:14.093216 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.093199    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 07 14:33:14.108200 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.108127    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"]
Jul 07 14:33:14.171186 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.171155    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbtlx\" (UniqueName: \"kubernetes.io/projected/e2046f4a-8260-418a-9e3a-9f091aa95324-kube-api-access-sbtlx\") pod \"lws-controller-manager-849574fc57-ff7t9\" (UID: \"e2046f4a-8260-418a-9e3a-9f091aa95324\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:33:14.171186 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.171195    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2046f4a-8260-418a-9e3a-9f091aa95324-cert\") pod \"lws-controller-manager-849574fc57-ff7t9\" (UID: \"e2046f4a-8260-418a-9e3a-9f091aa95324\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:33:14.171405 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.171231    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e2046f4a-8260-418a-9e3a-9f091aa95324-metrics-cert\") pod \"lws-controller-manager-849574fc57-ff7t9\" (UID: \"e2046f4a-8260-418a-9e3a-9f091aa95324\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:33:14.171405 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.171271    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e2046f4a-8260-418a-9e3a-9f091aa95324-manager-config\") pod \"lws-controller-manager-849574fc57-ff7t9\" (UID: \"e2046f4a-8260-418a-9e3a-9f091aa95324\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:33:14.272646 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.272605    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbtlx\" (UniqueName: \"kubernetes.io/projected/e2046f4a-8260-418a-9e3a-9f091aa95324-kube-api-access-sbtlx\") pod \"lws-controller-manager-849574fc57-ff7t9\" (UID: \"e2046f4a-8260-418a-9e3a-9f091aa95324\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:33:14.272646 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.272653    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2046f4a-8260-418a-9e3a-9f091aa95324-cert\") pod \"lws-controller-manager-849574fc57-ff7t9\" (UID: \"e2046f4a-8260-418a-9e3a-9f091aa95324\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:33:14.272890 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.272698    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e2046f4a-8260-418a-9e3a-9f091aa95324-metrics-cert\") pod \"lws-controller-manager-849574fc57-ff7t9\" (UID: \"e2046f4a-8260-418a-9e3a-9f091aa95324\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:33:14.272890 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.272741    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e2046f4a-8260-418a-9e3a-9f091aa95324-manager-config\") pod \"lws-controller-manager-849574fc57-ff7t9\" (UID: \"e2046f4a-8260-418a-9e3a-9f091aa95324\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:33:14.273401 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.273378    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e2046f4a-8260-418a-9e3a-9f091aa95324-manager-config\") pod \"lws-controller-manager-849574fc57-ff7t9\" (UID: \"e2046f4a-8260-418a-9e3a-9f091aa95324\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:33:14.275399 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.275377    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e2046f4a-8260-418a-9e3a-9f091aa95324-metrics-cert\") pod \"lws-controller-manager-849574fc57-ff7t9\" (UID: \"e2046f4a-8260-418a-9e3a-9f091aa95324\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:33:14.275508 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.275427    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2046f4a-8260-418a-9e3a-9f091aa95324-cert\") pod \"lws-controller-manager-849574fc57-ff7t9\" (UID: \"e2046f4a-8260-418a-9e3a-9f091aa95324\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:33:14.280226 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.280204    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbtlx\" (UniqueName: \"kubernetes.io/projected/e2046f4a-8260-418a-9e3a-9f091aa95324-kube-api-access-sbtlx\") pod \"lws-controller-manager-849574fc57-ff7t9\" (UID: \"e2046f4a-8260-418a-9e3a-9f091aa95324\") " pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:33:14.393658 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.393574    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:33:14.556853 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.553530    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"]
Jul 07 14:33:14.561345 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:33:14.561300    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2046f4a_8260_418a_9e3a_9f091aa95324.slice/crio-47b745f89d5359e4e453a10f48f189cfbd4434c760d1aa83434a1034982019f2 WatchSource:0}: Error finding container 47b745f89d5359e4e453a10f48f189cfbd4434c760d1aa83434a1034982019f2: Status 404 returned error can't find the container with id 47b745f89d5359e4e453a10f48f189cfbd4434c760d1aa83434a1034982019f2
Jul 07 14:33:14.658450 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:14.658330    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9" event={"ID":"e2046f4a-8260-418a-9e3a-9f091aa95324","Type":"ContainerStarted","Data":"47b745f89d5359e4e453a10f48f189cfbd4434c760d1aa83434a1034982019f2"}
Jul 07 14:33:15.648777 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:15.648744    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:16.859753 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:16.859707    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"]
Jul 07 14:33:16.860217 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:16.859989    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm" podUID="f5036107-8b95-4f86-a966-e14b6a5bd522" containerName="discovery" containerID="cri-o://008552afd4f6d836a8cb60703d39a76f5463156957f3f005dc7b84ee73ca2201" gracePeriod=30
Jul 07 14:33:17.115351 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.115285    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:17.305396 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.305350    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-csr-ca-configmap\") pod \"f5036107-8b95-4f86-a966-e14b6a5bd522\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") "
Jul 07 14:33:17.305396 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.305396    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-token\") pod \"f5036107-8b95-4f86-a966-e14b6a5bd522\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") "
Jul 07 14:33:17.305589 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.305428    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5036107-8b95-4f86-a966-e14b6a5bd522-local-certs\") pod \"f5036107-8b95-4f86-a966-e14b6a5bd522\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") "
Jul 07 14:33:17.305589 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.305472    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-csr-dns-cert\") pod \"f5036107-8b95-4f86-a966-e14b6a5bd522\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") "
Jul 07 14:33:17.305589 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.305548    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bk5q9\" (UniqueName: \"kubernetes.io/projected/f5036107-8b95-4f86-a966-e14b6a5bd522-kube-api-access-bk5q9\") pod \"f5036107-8b95-4f86-a966-e14b6a5bd522\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") "
Jul 07 14:33:17.305589 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.305583    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-cacerts\") pod \"f5036107-8b95-4f86-a966-e14b6a5bd522\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") "
Jul 07 14:33:17.305784 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.305604    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-kubeconfig\") pod \"f5036107-8b95-4f86-a966-e14b6a5bd522\" (UID: \"f5036107-8b95-4f86-a966-e14b6a5bd522\") "
Jul 07 14:33:17.305871 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.305830    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "f5036107-8b95-4f86-a966-e14b6a5bd522" (UID: "f5036107-8b95-4f86-a966-e14b6a5bd522"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 14:33:17.307805 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.307774    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5036107-8b95-4f86-a966-e14b6a5bd522-kube-api-access-bk5q9" (OuterVolumeSpecName: "kube-api-access-bk5q9") pod "f5036107-8b95-4f86-a966-e14b6a5bd522" (UID: "f5036107-8b95-4f86-a966-e14b6a5bd522"). InnerVolumeSpecName "kube-api-access-bk5q9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:33:17.307919 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.307801    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5036107-8b95-4f86-a966-e14b6a5bd522-local-certs" (OuterVolumeSpecName: "local-certs") pod "f5036107-8b95-4f86-a966-e14b6a5bd522" (UID: "f5036107-8b95-4f86-a966-e14b6a5bd522"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:33:17.308061 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.308002    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "f5036107-8b95-4f86-a966-e14b6a5bd522" (UID: "f5036107-8b95-4f86-a966-e14b6a5bd522"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:33:17.308061 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.308050    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-cacerts" (OuterVolumeSpecName: "cacerts") pod "f5036107-8b95-4f86-a966-e14b6a5bd522" (UID: "f5036107-8b95-4f86-a966-e14b6a5bd522"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:33:17.308137 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.308067    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "f5036107-8b95-4f86-a966-e14b6a5bd522" (UID: "f5036107-8b95-4f86-a966-e14b6a5bd522"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:33:17.308137 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.308113    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-token" (OuterVolumeSpecName: "istio-token") pod "f5036107-8b95-4f86-a966-e14b6a5bd522" (UID: "f5036107-8b95-4f86-a966-e14b6a5bd522"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:33:17.407061 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.406933    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bk5q9\" (UniqueName: \"kubernetes.io/projected/f5036107-8b95-4f86-a966-e14b6a5bd522-kube-api-access-bk5q9\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:33:17.407061 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.406967    2583 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-cacerts\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:33:17.407061 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.407001    2583 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-kubeconfig\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:33:17.407061 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.407012    2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-csr-ca-configmap\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:33:17.407061 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.407022    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-token\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:33:17.407061 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.407030    2583 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5036107-8b95-4f86-a966-e14b6a5bd522-local-certs\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:33:17.407061 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.407037    2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f5036107-8b95-4f86-a966-e14b6a5bd522-istio-csr-dns-cert\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:33:17.674654 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.674554    2583 generic.go:356] "Generic (PLEG): container finished" podID="f5036107-8b95-4f86-a966-e14b6a5bd522" containerID="008552afd4f6d836a8cb60703d39a76f5463156957f3f005dc7b84ee73ca2201" exitCode=0
Jul 07 14:33:17.674654 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.674598    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"
Jul 07 14:33:17.674854 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.674590    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm" event={"ID":"f5036107-8b95-4f86-a966-e14b6a5bd522","Type":"ContainerDied","Data":"008552afd4f6d836a8cb60703d39a76f5463156957f3f005dc7b84ee73ca2201"}
Jul 07 14:33:17.674854 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.674724    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm" event={"ID":"f5036107-8b95-4f86-a966-e14b6a5bd522","Type":"ContainerDied","Data":"616d997c339d37b8d6da0a7efb7dba39a10606c989edd5724724253ae2f85704"}
Jul 07 14:33:17.674854 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.674754    2583 scope.go:117] "RemoveContainer" containerID="008552afd4f6d836a8cb60703d39a76f5463156957f3f005dc7b84ee73ca2201"
Jul 07 14:33:17.676317 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.676285    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9" event={"ID":"e2046f4a-8260-418a-9e3a-9f091aa95324","Type":"ContainerStarted","Data":"369258ae8ce6843e6e9d835c47536552073dc90b41b753a58f5ac3f1c679527c"}
Jul 07 14:33:17.676428 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.676410    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:33:17.683828 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.683810    2583 scope.go:117] "RemoveContainer" containerID="008552afd4f6d836a8cb60703d39a76f5463156957f3f005dc7b84ee73ca2201"
Jul 07 14:33:17.684141 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:33:17.684122    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"008552afd4f6d836a8cb60703d39a76f5463156957f3f005dc7b84ee73ca2201\": container with ID starting with 008552afd4f6d836a8cb60703d39a76f5463156957f3f005dc7b84ee73ca2201 not found: ID does not exist" containerID="008552afd4f6d836a8cb60703d39a76f5463156957f3f005dc7b84ee73ca2201"
Jul 07 14:33:17.684189 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.684152    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"008552afd4f6d836a8cb60703d39a76f5463156957f3f005dc7b84ee73ca2201"} err="failed to get container status \"008552afd4f6d836a8cb60703d39a76f5463156957f3f005dc7b84ee73ca2201\": rpc error: code = NotFound desc = could not find container \"008552afd4f6d836a8cb60703d39a76f5463156957f3f005dc7b84ee73ca2201\": container with ID starting with 008552afd4f6d836a8cb60703d39a76f5463156957f3f005dc7b84ee73ca2201 not found: ID does not exist"
Jul 07 14:33:17.695659 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.695613    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9" podStartSLOduration=1.2950621039999999 podStartE2EDuration="3.695595327s" totalImagesPullingTime="2.400533223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:33:17.694659521 +0000 UTC m=+412.157559464" watchObservedRunningTime="2026-07-07 14:33:17.695595327 +0000 UTC m=+412.158495313"
Jul 07 14:33:17.711784 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.711754    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"]
Jul 07 14:33:17.717628 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:17.717605    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2bvxm"]
Jul 07 14:33:18.100592 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:18.100561    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5036107-8b95-4f86-a966-e14b6a5bd522" path="/var/lib/kubelet/pods/f5036107-8b95-4f86-a966-e14b6a5bd522/volumes"
Jul 07 14:33:22.650528 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:22.650494    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-fq86n"
Jul 07 14:33:22.652522 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:22.652505    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-cm5cr"
Jul 07 14:33:28.683782 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:33:28.683747    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-849574fc57-ff7t9"
Jul 07 14:34:53.942812 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:53.942778    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln"]
Jul 07 14:34:53.943308 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:53.943087    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5036107-8b95-4f86-a966-e14b6a5bd522" containerName="discovery"
Jul 07 14:34:53.943308 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:53.943098    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5036107-8b95-4f86-a966-e14b6a5bd522" containerName="discovery"
Jul 07 14:34:53.943308 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:53.943147    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5036107-8b95-4f86-a966-e14b6a5bd522" containerName="discovery"
Jul 07 14:34:53.945898 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:53.945881    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln"
Jul 07 14:34:53.948106 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:53.948079    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 07 14:34:53.948257 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:53.948149    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 07 14:34:53.948257 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:53.948233    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gbxt7\""
Jul 07 14:34:53.948809 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:53.948792    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 14:34:53.948909 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:53.948835    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 14:34:53.958810 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:53.958784    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln"]
Jul 07 14:34:54.073863 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:54.073830    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/edf0a916-c32a-47a4-a4dc-2a12e37b0d3e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-qg4ln\" (UID: \"edf0a916-c32a-47a4-a4dc-2a12e37b0d3e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln"
Jul 07 14:34:54.073863 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:54.073862    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/edf0a916-c32a-47a4-a4dc-2a12e37b0d3e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-qg4ln\" (UID: \"edf0a916-c32a-47a4-a4dc-2a12e37b0d3e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln"
Jul 07 14:34:54.074114 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:54.073902    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8glv\" (UniqueName: \"kubernetes.io/projected/edf0a916-c32a-47a4-a4dc-2a12e37b0d3e-kube-api-access-l8glv\") pod \"kuadrant-console-plugin-66b5774985-qg4ln\" (UID: \"edf0a916-c32a-47a4-a4dc-2a12e37b0d3e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln"
Jul 07 14:34:54.174483 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:54.174434    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/edf0a916-c32a-47a4-a4dc-2a12e37b0d3e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-qg4ln\" (UID: \"edf0a916-c32a-47a4-a4dc-2a12e37b0d3e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln"
Jul 07 14:34:54.174483 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:54.174488    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/edf0a916-c32a-47a4-a4dc-2a12e37b0d3e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-qg4ln\" (UID: \"edf0a916-c32a-47a4-a4dc-2a12e37b0d3e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln"
Jul 07 14:34:54.174706 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:54.174536    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8glv\" (UniqueName: \"kubernetes.io/projected/edf0a916-c32a-47a4-a4dc-2a12e37b0d3e-kube-api-access-l8glv\") pod \"kuadrant-console-plugin-66b5774985-qg4ln\" (UID: \"edf0a916-c32a-47a4-a4dc-2a12e37b0d3e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln"
Jul 07 14:34:54.175176 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:54.175153    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/edf0a916-c32a-47a4-a4dc-2a12e37b0d3e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-qg4ln\" (UID: \"edf0a916-c32a-47a4-a4dc-2a12e37b0d3e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln"
Jul 07 14:34:54.176888 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:54.176868    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/edf0a916-c32a-47a4-a4dc-2a12e37b0d3e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-qg4ln\" (UID: \"edf0a916-c32a-47a4-a4dc-2a12e37b0d3e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln"
Jul 07 14:34:54.184743 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:54.184706    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8glv\" (UniqueName: \"kubernetes.io/projected/edf0a916-c32a-47a4-a4dc-2a12e37b0d3e-kube-api-access-l8glv\") pod \"kuadrant-console-plugin-66b5774985-qg4ln\" (UID: \"edf0a916-c32a-47a4-a4dc-2a12e37b0d3e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln"
Jul 07 14:34:54.255100 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:54.255009    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln"
Jul 07 14:34:54.388577 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:54.388505    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln"]
Jul 07 14:34:54.391219 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:34:54.391192    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podedf0a916_c32a_47a4_a4dc_2a12e37b0d3e.slice/crio-2a6925cf33db3c682429f0708fa884e6a1f4afd288abda3e340287e70d9499bf WatchSource:0}: Error finding container 2a6925cf33db3c682429f0708fa884e6a1f4afd288abda3e340287e70d9499bf: Status 404 returned error can't find the container with id 2a6925cf33db3c682429f0708fa884e6a1f4afd288abda3e340287e70d9499bf
Jul 07 14:34:55.072804 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:34:55.072770    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln" event={"ID":"edf0a916-c32a-47a4-a4dc-2a12e37b0d3e","Type":"ContainerStarted","Data":"2a6925cf33db3c682429f0708fa884e6a1f4afd288abda3e340287e70d9499bf"}
Jul 07 14:35:19.198314 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:19.198277    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln" event={"ID":"edf0a916-c32a-47a4-a4dc-2a12e37b0d3e","Type":"ContainerStarted","Data":"6c5a92400782bcb9b256447544c75a4a14fdc364869deb92962975ceddb828ee"}
Jul 07 14:35:19.215359 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:19.215312    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qg4ln" podStartSLOduration=2.091281977 podStartE2EDuration="26.215296692s" totalImagesPullingTime="24.124014715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:34:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:35:19.213353373 +0000 UTC m=+533.676253316" watchObservedRunningTime="2026-07-07 14:35:19.215296692 +0000 UTC m=+533.678196633"
Jul 07 14:35:39.176625 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.176588    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:35:39.201316 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.201288    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:35:39.201463 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.201382    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-tpftx"
Jul 07 14:35:39.203297 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.203274    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 07 14:35:39.211552 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.211530    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:35:39.249898 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.249872    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/012f69d1-b881-4733-af1b-d899441d50e9-config-file\") pod \"limitador-limitador-54d6d988d5-tpftx\" (UID: \"012f69d1-b881-4733-af1b-d899441d50e9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tpftx"
Jul 07 14:35:39.250086 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.249913    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpls9\" (UniqueName: \"kubernetes.io/projected/012f69d1-b881-4733-af1b-d899441d50e9-kube-api-access-mpls9\") pod \"limitador-limitador-54d6d988d5-tpftx\" (UID: \"012f69d1-b881-4733-af1b-d899441d50e9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tpftx"
Jul 07 14:35:39.350436 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.350394    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/012f69d1-b881-4733-af1b-d899441d50e9-config-file\") pod \"limitador-limitador-54d6d988d5-tpftx\" (UID: \"012f69d1-b881-4733-af1b-d899441d50e9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tpftx"
Jul 07 14:35:39.350597 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.350468    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpls9\" (UniqueName: \"kubernetes.io/projected/012f69d1-b881-4733-af1b-d899441d50e9-kube-api-access-mpls9\") pod \"limitador-limitador-54d6d988d5-tpftx\" (UID: \"012f69d1-b881-4733-af1b-d899441d50e9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tpftx"
Jul 07 14:35:39.350971 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.350953    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/012f69d1-b881-4733-af1b-d899441d50e9-config-file\") pod \"limitador-limitador-54d6d988d5-tpftx\" (UID: \"012f69d1-b881-4733-af1b-d899441d50e9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tpftx"
Jul 07 14:35:39.358206 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.358183    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpls9\" (UniqueName: \"kubernetes.io/projected/012f69d1-b881-4733-af1b-d899441d50e9-kube-api-access-mpls9\") pod \"limitador-limitador-54d6d988d5-tpftx\" (UID: \"012f69d1-b881-4733-af1b-d899441d50e9\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-tpftx"
Jul 07 14:35:39.511744 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.511712    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-tpftx"
Jul 07 14:35:39.652413 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.652387    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:35:39.655113 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:35:39.655082    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod012f69d1_b881_4733_af1b_d899441d50e9.slice/crio-670745cf9428c722003ca25e6c28073286c14a53ff9e39ffc1716d8adc2c979e WatchSource:0}: Error finding container 670745cf9428c722003ca25e6c28073286c14a53ff9e39ffc1716d8adc2c979e: Status 404 returned error can't find the container with id 670745cf9428c722003ca25e6c28073286c14a53ff9e39ffc1716d8adc2c979e
Jul 07 14:35:39.768903 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.768823    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-66fwq"]
Jul 07 14:35:39.773183 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.773163    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-66fwq"
Jul 07 14:35:39.775332 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.775307    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d8cm8\""
Jul 07 14:35:39.780298 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.780085    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-66fwq"]
Jul 07 14:35:39.854007 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.853943    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2zlw\" (UniqueName: \"kubernetes.io/projected/e9eb8ab7-a97b-4840-a4ce-b5f9462ff039-kube-api-access-l2zlw\") pod \"authorino-75d556bd77-66fwq\" (UID: \"e9eb8ab7-a97b-4840-a4ce-b5f9462ff039\") " pod="kuadrant-system/authorino-75d556bd77-66fwq"
Jul 07 14:35:39.954847 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.954812    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2zlw\" (UniqueName: \"kubernetes.io/projected/e9eb8ab7-a97b-4840-a4ce-b5f9462ff039-kube-api-access-l2zlw\") pod \"authorino-75d556bd77-66fwq\" (UID: \"e9eb8ab7-a97b-4840-a4ce-b5f9462ff039\") " pod="kuadrant-system/authorino-75d556bd77-66fwq"
Jul 07 14:35:39.962572 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:39.962537    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2zlw\" (UniqueName: \"kubernetes.io/projected/e9eb8ab7-a97b-4840-a4ce-b5f9462ff039-kube-api-access-l2zlw\") pod \"authorino-75d556bd77-66fwq\" (UID: \"e9eb8ab7-a97b-4840-a4ce-b5f9462ff039\") " pod="kuadrant-system/authorino-75d556bd77-66fwq"
Jul 07 14:35:40.083490 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:40.083367    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-66fwq"
Jul 07 14:35:40.267210 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:40.267138    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-66fwq"]
Jul 07 14:35:40.277711 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:35:40.277665    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9eb8ab7_a97b_4840_a4ce_b5f9462ff039.slice/crio-9574ce7388762a1ccd5bc5609fa1508ac72320eec9c39a828f488ad3bbbc49fc WatchSource:0}: Error finding container 9574ce7388762a1ccd5bc5609fa1508ac72320eec9c39a828f488ad3bbbc49fc: Status 404 returned error can't find the container with id 9574ce7388762a1ccd5bc5609fa1508ac72320eec9c39a828f488ad3bbbc49fc
Jul 07 14:35:40.286137 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:40.286055    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-tpftx" event={"ID":"012f69d1-b881-4733-af1b-d899441d50e9","Type":"ContainerStarted","Data":"670745cf9428c722003ca25e6c28073286c14a53ff9e39ffc1716d8adc2c979e"}
Jul 07 14:35:41.292504 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:41.292466    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-66fwq" event={"ID":"e9eb8ab7-a97b-4840-a4ce-b5f9462ff039","Type":"ContainerStarted","Data":"9574ce7388762a1ccd5bc5609fa1508ac72320eec9c39a828f488ad3bbbc49fc"}
Jul 07 14:35:43.304740 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:43.304693    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-tpftx" event={"ID":"012f69d1-b881-4733-af1b-d899441d50e9","Type":"ContainerStarted","Data":"92705d16eb5a18ae586ea042acc07007840f2945b4bcaa9b9ea171d81fbdbceb"}
Jul 07 14:35:43.305156 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:43.304792    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-tpftx"
Jul 07 14:35:43.323616 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:43.323558    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-tpftx" podStartSLOduration=1.7315941750000001 podStartE2EDuration="4.323538142s" totalImagesPullingTime="2.591943967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:35:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:35:43.322538787 +0000 UTC m=+557.785438730" watchObservedRunningTime="2026-07-07 14:35:43.323538142 +0000 UTC m=+557.786438086"
Jul 07 14:35:45.314388 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:45.314352    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-66fwq" event={"ID":"e9eb8ab7-a97b-4840-a4ce-b5f9462ff039","Type":"ContainerStarted","Data":"a597ab80dbf4dd3aa287272170c1bb14afdced64205b99b622a705906ec11695"}
Jul 07 14:35:54.311240 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:54.311207    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-tpftx"
Jul 07 14:35:54.329460 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:35:54.329410    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-66fwq" podStartSLOduration=11.179355537 podStartE2EDuration="15.329391833s" totalImagesPullingTime="4.150036296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:35:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:35:45.32904336 +0000 UTC m=+559.791943303" watchObservedRunningTime="2026-07-07 14:35:54.329391833 +0000 UTC m=+568.792291777"
Jul 07 14:36:12.717616 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:12.717583    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d6996576b-dg4bw"]
Jul 07 14:36:12.720852 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:12.720832    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d6996576b-dg4bw"
Jul 07 14:36:12.723709 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:12.723690    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 07 14:36:12.736621 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:12.736599    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d6996576b-dg4bw"]
Jul 07 14:36:12.813398 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:12.813355    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad48c566-1ca3-4467-8c51-e62be1fabddc-tls-cert\") pod \"authorino-6d6996576b-dg4bw\" (UID: \"ad48c566-1ca3-4467-8c51-e62be1fabddc\") " pod="kuadrant-system/authorino-6d6996576b-dg4bw"
Jul 07 14:36:12.813576 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:12.813420    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf4lw\" (UniqueName: \"kubernetes.io/projected/ad48c566-1ca3-4467-8c51-e62be1fabddc-kube-api-access-kf4lw\") pod \"authorino-6d6996576b-dg4bw\" (UID: \"ad48c566-1ca3-4467-8c51-e62be1fabddc\") " pod="kuadrant-system/authorino-6d6996576b-dg4bw"
Jul 07 14:36:12.914854 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:12.914801    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf4lw\" (UniqueName: \"kubernetes.io/projected/ad48c566-1ca3-4467-8c51-e62be1fabddc-kube-api-access-kf4lw\") pod \"authorino-6d6996576b-dg4bw\" (UID: \"ad48c566-1ca3-4467-8c51-e62be1fabddc\") " pod="kuadrant-system/authorino-6d6996576b-dg4bw"
Jul 07 14:36:12.915035 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:12.914901    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad48c566-1ca3-4467-8c51-e62be1fabddc-tls-cert\") pod \"authorino-6d6996576b-dg4bw\" (UID: \"ad48c566-1ca3-4467-8c51-e62be1fabddc\") " pod="kuadrant-system/authorino-6d6996576b-dg4bw"
Jul 07 14:36:12.917352 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:12.917323    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad48c566-1ca3-4467-8c51-e62be1fabddc-tls-cert\") pod \"authorino-6d6996576b-dg4bw\" (UID: \"ad48c566-1ca3-4467-8c51-e62be1fabddc\") " pod="kuadrant-system/authorino-6d6996576b-dg4bw"
Jul 07 14:36:12.922400 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:12.922377    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf4lw\" (UniqueName: \"kubernetes.io/projected/ad48c566-1ca3-4467-8c51-e62be1fabddc-kube-api-access-kf4lw\") pod \"authorino-6d6996576b-dg4bw\" (UID: \"ad48c566-1ca3-4467-8c51-e62be1fabddc\") " pod="kuadrant-system/authorino-6d6996576b-dg4bw"
Jul 07 14:36:13.029238 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:13.029157    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d6996576b-dg4bw"
Jul 07 14:36:13.161267 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:13.161112    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d6996576b-dg4bw"]
Jul 07 14:36:13.163953 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:36:13.163923    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad48c566_1ca3_4467_8c51_e62be1fabddc.slice/crio-e3d552b8ca926bc8cc8a55849b4f8d4ace68c2048a1d681759b9ea6f4479c80a WatchSource:0}: Error finding container e3d552b8ca926bc8cc8a55849b4f8d4ace68c2048a1d681759b9ea6f4479c80a: Status 404 returned error can't find the container with id e3d552b8ca926bc8cc8a55849b4f8d4ace68c2048a1d681759b9ea6f4479c80a
Jul 07 14:36:13.431280 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:13.431249    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d6996576b-dg4bw" event={"ID":"ad48c566-1ca3-4467-8c51-e62be1fabddc","Type":"ContainerStarted","Data":"e3d552b8ca926bc8cc8a55849b4f8d4ace68c2048a1d681759b9ea6f4479c80a"}
Jul 07 14:36:14.437070 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:14.437033    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d6996576b-dg4bw" event={"ID":"ad48c566-1ca3-4467-8c51-e62be1fabddc","Type":"ContainerStarted","Data":"84c825877221a22b36b1914b79d7616d6db6ea34cd96c6bd4a7c56332b2e099a"}
Jul 07 14:36:14.453684 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:14.453634    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d6996576b-dg4bw" podStartSLOduration=2.080373283 podStartE2EDuration="2.453615575s" totalImagesPullingTime="373.242292ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:36:14.452597099 +0000 UTC m=+588.915497055" watchObservedRunningTime="2026-07-07 14:36:14.453615575 +0000 UTC m=+588.916515518"
Jul 07 14:36:14.479627 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:14.479592    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-66fwq"]
Jul 07 14:36:14.479803 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:14.479774    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-66fwq" podUID="e9eb8ab7-a97b-4840-a4ce-b5f9462ff039" containerName="authorino" containerID="cri-o://a597ab80dbf4dd3aa287272170c1bb14afdced64205b99b622a705906ec11695" gracePeriod=30
Jul 07 14:36:14.774169 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:14.774136    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-66fwq"
Jul 07 14:36:14.830921 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:14.830885    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2zlw\" (UniqueName: \"kubernetes.io/projected/e9eb8ab7-a97b-4840-a4ce-b5f9462ff039-kube-api-access-l2zlw\") pod \"e9eb8ab7-a97b-4840-a4ce-b5f9462ff039\" (UID: \"e9eb8ab7-a97b-4840-a4ce-b5f9462ff039\") "
Jul 07 14:36:14.833136 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:14.833101    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9eb8ab7-a97b-4840-a4ce-b5f9462ff039-kube-api-access-l2zlw" (OuterVolumeSpecName: "kube-api-access-l2zlw") pod "e9eb8ab7-a97b-4840-a4ce-b5f9462ff039" (UID: "e9eb8ab7-a97b-4840-a4ce-b5f9462ff039"). InnerVolumeSpecName "kube-api-access-l2zlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:36:14.931912 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:14.931872    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2zlw\" (UniqueName: \"kubernetes.io/projected/e9eb8ab7-a97b-4840-a4ce-b5f9462ff039-kube-api-access-l2zlw\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:36:15.442688 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:15.442653    2583 generic.go:356] "Generic (PLEG): container finished" podID="e9eb8ab7-a97b-4840-a4ce-b5f9462ff039" containerID="a597ab80dbf4dd3aa287272170c1bb14afdced64205b99b622a705906ec11695" exitCode=0
Jul 07 14:36:15.443162 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:15.442696    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-66fwq"
Jul 07 14:36:15.443162 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:15.442738    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-66fwq" event={"ID":"e9eb8ab7-a97b-4840-a4ce-b5f9462ff039","Type":"ContainerDied","Data":"a597ab80dbf4dd3aa287272170c1bb14afdced64205b99b622a705906ec11695"}
Jul 07 14:36:15.443162 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:15.442775    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-66fwq" event={"ID":"e9eb8ab7-a97b-4840-a4ce-b5f9462ff039","Type":"ContainerDied","Data":"9574ce7388762a1ccd5bc5609fa1508ac72320eec9c39a828f488ad3bbbc49fc"}
Jul 07 14:36:15.443162 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:15.442793    2583 scope.go:117] "RemoveContainer" containerID="a597ab80dbf4dd3aa287272170c1bb14afdced64205b99b622a705906ec11695"
Jul 07 14:36:15.451733 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:15.451710    2583 scope.go:117] "RemoveContainer" containerID="a597ab80dbf4dd3aa287272170c1bb14afdced64205b99b622a705906ec11695"
Jul 07 14:36:15.452056 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:36:15.452036    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a597ab80dbf4dd3aa287272170c1bb14afdced64205b99b622a705906ec11695\": container with ID starting with a597ab80dbf4dd3aa287272170c1bb14afdced64205b99b622a705906ec11695 not found: ID does not exist" containerID="a597ab80dbf4dd3aa287272170c1bb14afdced64205b99b622a705906ec11695"
Jul 07 14:36:15.452110 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:15.452066    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a597ab80dbf4dd3aa287272170c1bb14afdced64205b99b622a705906ec11695"} err="failed to get container status \"a597ab80dbf4dd3aa287272170c1bb14afdced64205b99b622a705906ec11695\": rpc error: code = NotFound desc = could not find container \"a597ab80dbf4dd3aa287272170c1bb14afdced64205b99b622a705906ec11695\": container with ID starting with a597ab80dbf4dd3aa287272170c1bb14afdced64205b99b622a705906ec11695 not found: ID does not exist"
Jul 07 14:36:15.462381 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:15.462349    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-66fwq"]
Jul 07 14:36:15.467468 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:15.467439    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-66fwq"]
Jul 07 14:36:16.100476 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:16.100436    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9eb8ab7-a97b-4840-a4ce-b5f9462ff039" path="/var/lib/kubelet/pods/e9eb8ab7-a97b-4840-a4ce-b5f9462ff039/volumes"
Jul 07 14:36:25.990462 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:25.990433    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:36:25.994941 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:25.994920    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:36:53.186086 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:36:53.186008    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:37:32.580237 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:37:32.580202    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:37:48.144856 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:37:48.144813    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:38:05.637100 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:05.637056    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:38:10.035375 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:10.035330    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:38:38.448944 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.448906    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"]
Jul 07 14:38:38.449417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.449279    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9eb8ab7-a97b-4840-a4ce-b5f9462ff039" containerName="authorino"
Jul 07 14:38:38.449417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.449292    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9eb8ab7-a97b-4840-a4ce-b5f9462ff039" containerName="authorino"
Jul 07 14:38:38.449417 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.449340    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9eb8ab7-a97b-4840-a4ce-b5f9462ff039" containerName="authorino"
Jul 07 14:38:38.452166 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.452150    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.454474 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.454452    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 14:38:38.454596 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.454452    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wvkhz\""
Jul 07 14:38:38.454950 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.454935    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 14:38:38.455036 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.455009    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 07 14:38:38.464494 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.464473    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"]
Jul 07 14:38:38.550237 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.550198    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp7hz\" (UniqueName: \"kubernetes.io/projected/0a4806ad-14e7-4f78-b807-9c9853d1f18c-kube-api-access-kp7hz\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.550402 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.550242    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a4806ad-14e7-4f78-b807-9c9853d1f18c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.550402 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.550278    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a4806ad-14e7-4f78-b807-9c9853d1f18c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.550402 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.550360    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a4806ad-14e7-4f78-b807-9c9853d1f18c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.550530 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.550413    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a4806ad-14e7-4f78-b807-9c9853d1f18c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.550530 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.550451    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a4806ad-14e7-4f78-b807-9c9853d1f18c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.651541 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.651492    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp7hz\" (UniqueName: \"kubernetes.io/projected/0a4806ad-14e7-4f78-b807-9c9853d1f18c-kube-api-access-kp7hz\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.651541 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.651545    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a4806ad-14e7-4f78-b807-9c9853d1f18c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.651790 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.651655    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a4806ad-14e7-4f78-b807-9c9853d1f18c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.651790 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.651734    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a4806ad-14e7-4f78-b807-9c9853d1f18c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.651790 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.651778    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a4806ad-14e7-4f78-b807-9c9853d1f18c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.651925 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.651806    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a4806ad-14e7-4f78-b807-9c9853d1f18c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.651925 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.651864    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a4806ad-14e7-4f78-b807-9c9853d1f18c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.652043 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.652023    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a4806ad-14e7-4f78-b807-9c9853d1f18c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.652103 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.652089    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a4806ad-14e7-4f78-b807-9c9853d1f18c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.654116 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.654093    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a4806ad-14e7-4f78-b807-9c9853d1f18c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.654356 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.654339    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a4806ad-14e7-4f78-b807-9c9853d1f18c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.659483 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.659455    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp7hz\" (UniqueName: \"kubernetes.io/projected/0a4806ad-14e7-4f78-b807-9c9853d1f18c-kube-api-access-kp7hz\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74\" (UID: \"0a4806ad-14e7-4f78-b807-9c9853d1f18c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.760718 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.760617    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:38.893794 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.893755    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"]
Jul 07 14:38:38.897232 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:38:38.897203    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a4806ad_14e7_4f78_b807_9c9853d1f18c.slice/crio-e06a2654235d0dae8e6470ee2fafd38f88bcbc370d0ecad585a76f8c94391788 WatchSource:0}: Error finding container e06a2654235d0dae8e6470ee2fafd38f88bcbc370d0ecad585a76f8c94391788: Status 404 returned error can't find the container with id e06a2654235d0dae8e6470ee2fafd38f88bcbc370d0ecad585a76f8c94391788
Jul 07 14:38:38.898526 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:38.898507    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 14:38:39.035618 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:39.035530    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74" event={"ID":"0a4806ad-14e7-4f78-b807-9c9853d1f18c","Type":"ContainerStarted","Data":"e06a2654235d0dae8e6470ee2fafd38f88bcbc370d0ecad585a76f8c94391788"}
Jul 07 14:38:40.041681 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:40.041644    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74" event={"ID":"0a4806ad-14e7-4f78-b807-9c9853d1f18c","Type":"ContainerStarted","Data":"e4a7f9ee597ddb7eddeb330931cf66b48396e83d0a1792896d89ba534c449e2c"}
Jul 07 14:38:40.066143 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:40.066090    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74" podStartSLOduration=0.99828979 podStartE2EDuration="2.066073954s" totalImagesPullingTime="1.067784164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:38:40.063652415 +0000 UTC m=+734.526552370" watchObservedRunningTime="2026-07-07 14:38:40.066073954 +0000 UTC m=+734.528973897"
Jul 07 14:38:40.335574 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:40.335483    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:38:48.761629 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:48.761586    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:48.762059 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:48.761646    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:48.773940 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:48.773913    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:49.089950 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:49.089867    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74"
Jul 07 14:38:57.648610 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:38:57.648577    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:39:59.873064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:39:59.872967    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d8694fd4c-jp4mj"]
Jul 07 14:39:59.876182 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:39:59.876165    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8694fd4c-jp4mj"
Jul 07 14:39:59.886934 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:39:59.886907    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d8694fd4c-jp4mj"]
Jul 07 14:40:00.031317 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:00.031282    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ee076dc5-aed4-45c6-b138-7691c0b183ce-tls-cert\") pod \"authorino-d8694fd4c-jp4mj\" (UID: \"ee076dc5-aed4-45c6-b138-7691c0b183ce\") " pod="kuadrant-system/authorino-d8694fd4c-jp4mj"
Jul 07 14:40:00.031497 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:00.031354    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9hxc\" (UniqueName: \"kubernetes.io/projected/ee076dc5-aed4-45c6-b138-7691c0b183ce-kube-api-access-p9hxc\") pod \"authorino-d8694fd4c-jp4mj\" (UID: \"ee076dc5-aed4-45c6-b138-7691c0b183ce\") " pod="kuadrant-system/authorino-d8694fd4c-jp4mj"
Jul 07 14:40:00.132789 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:00.132703    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9hxc\" (UniqueName: \"kubernetes.io/projected/ee076dc5-aed4-45c6-b138-7691c0b183ce-kube-api-access-p9hxc\") pod \"authorino-d8694fd4c-jp4mj\" (UID: \"ee076dc5-aed4-45c6-b138-7691c0b183ce\") " pod="kuadrant-system/authorino-d8694fd4c-jp4mj"
Jul 07 14:40:00.132789 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:00.132757    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ee076dc5-aed4-45c6-b138-7691c0b183ce-tls-cert\") pod \"authorino-d8694fd4c-jp4mj\" (UID: \"ee076dc5-aed4-45c6-b138-7691c0b183ce\") " pod="kuadrant-system/authorino-d8694fd4c-jp4mj"
Jul 07 14:40:00.135255 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:00.135223    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ee076dc5-aed4-45c6-b138-7691c0b183ce-tls-cert\") pod \"authorino-d8694fd4c-jp4mj\" (UID: \"ee076dc5-aed4-45c6-b138-7691c0b183ce\") " pod="kuadrant-system/authorino-d8694fd4c-jp4mj"
Jul 07 14:40:00.140444 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:00.140418    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9hxc\" (UniqueName: \"kubernetes.io/projected/ee076dc5-aed4-45c6-b138-7691c0b183ce-kube-api-access-p9hxc\") pod \"authorino-d8694fd4c-jp4mj\" (UID: \"ee076dc5-aed4-45c6-b138-7691c0b183ce\") " pod="kuadrant-system/authorino-d8694fd4c-jp4mj"
Jul 07 14:40:00.185023 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:00.184964    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d8694fd4c-jp4mj"
Jul 07 14:40:00.324037 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:00.324010    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d8694fd4c-jp4mj"]
Jul 07 14:40:00.326607 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:40:00.326579    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee076dc5_aed4_45c6_b138_7691c0b183ce.slice/crio-7bdba095cd70333dbd16e6e518c01fc4570508e5757da678b1b697e18db4feec WatchSource:0}: Error finding container 7bdba095cd70333dbd16e6e518c01fc4570508e5757da678b1b697e18db4feec: Status 404 returned error can't find the container with id 7bdba095cd70333dbd16e6e518c01fc4570508e5757da678b1b697e18db4feec
Jul 07 14:40:00.367232 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:00.367187    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8694fd4c-jp4mj" event={"ID":"ee076dc5-aed4-45c6-b138-7691c0b183ce","Type":"ContainerStarted","Data":"7bdba095cd70333dbd16e6e518c01fc4570508e5757da678b1b697e18db4feec"}
Jul 07 14:40:00.820213 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:40:00.820173    2583 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="quay.io/kuadrant/authorino:v0.24.0"
Jul 07 14:40:00.820411 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:40:00.820296    2583 kuberuntime_manager.go:1459] "Unhandled Error" err="container authorino start failed in pod authorino-d8694fd4c-jp4mj_kuadrant-system(ee076dc5-aed4-45c6-b138-7691c0b183ce): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" logger="UnhandledError"
Jul 07 14:40:00.820411 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:40:00.820338    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"authorino\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="kuadrant-system/authorino-d8694fd4c-jp4mj" podUID="ee076dc5-aed4-45c6-b138-7691c0b183ce"
Jul 07 14:40:01.372080 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:40:01.372043    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"authorino\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/kuadrant/authorino:v0.24.0\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="kuadrant-system/authorino-d8694fd4c-jp4mj" podUID="ee076dc5-aed4-45c6-b138-7691c0b183ce"
Jul 07 14:40:17.447078 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:17.447028    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d8694fd4c-jp4mj" event={"ID":"ee076dc5-aed4-45c6-b138-7691c0b183ce","Type":"ContainerStarted","Data":"1ffb634df23da6b5dfef6ba05738d57b6c8425c307ea63d6734060a5963e183d"}
Jul 07 14:40:17.497881 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:17.497816    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d8694fd4c-jp4mj" podStartSLOduration=2.310554779 podStartE2EDuration="18.497794462s" totalImagesPullingTime="16.187239683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:39:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-07 14:40:17.467758925 +0000 UTC m=+831.930658870" watchObservedRunningTime="2026-07-07 14:40:17.497794462 +0000 UTC m=+831.960694467"
Jul 07 14:40:17.498643 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:17.498621    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d6996576b-dg4bw"]
Jul 07 14:40:17.498814 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:17.498796    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6d6996576b-dg4bw" podUID="ad48c566-1ca3-4467-8c51-e62be1fabddc" containerName="authorino" containerID="cri-o://84c825877221a22b36b1914b79d7616d6db6ea34cd96c6bd4a7c56332b2e099a" gracePeriod=30
Jul 07 14:40:17.759777 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:17.759750    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d6996576b-dg4bw"
Jul 07 14:40:17.785727 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:17.785698    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad48c566-1ca3-4467-8c51-e62be1fabddc-tls-cert\") pod \"ad48c566-1ca3-4467-8c51-e62be1fabddc\" (UID: \"ad48c566-1ca3-4467-8c51-e62be1fabddc\") "
Jul 07 14:40:17.796342 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:17.796308    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad48c566-1ca3-4467-8c51-e62be1fabddc-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ad48c566-1ca3-4467-8c51-e62be1fabddc" (UID: "ad48c566-1ca3-4467-8c51-e62be1fabddc"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:40:17.886804 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:17.886768    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kf4lw\" (UniqueName: \"kubernetes.io/projected/ad48c566-1ca3-4467-8c51-e62be1fabddc-kube-api-access-kf4lw\") pod \"ad48c566-1ca3-4467-8c51-e62be1fabddc\" (UID: \"ad48c566-1ca3-4467-8c51-e62be1fabddc\") "
Jul 07 14:40:17.886966 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:17.886953    2583 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad48c566-1ca3-4467-8c51-e62be1fabddc-tls-cert\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:40:17.888795 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:17.888765    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad48c566-1ca3-4467-8c51-e62be1fabddc-kube-api-access-kf4lw" (OuterVolumeSpecName: "kube-api-access-kf4lw") pod "ad48c566-1ca3-4467-8c51-e62be1fabddc" (UID: "ad48c566-1ca3-4467-8c51-e62be1fabddc"). InnerVolumeSpecName "kube-api-access-kf4lw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:40:17.988241 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:17.988159    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kf4lw\" (UniqueName: \"kubernetes.io/projected/ad48c566-1ca3-4467-8c51-e62be1fabddc-kube-api-access-kf4lw\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:40:18.452247 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:18.452217    2583 generic.go:356] "Generic (PLEG): container finished" podID="ad48c566-1ca3-4467-8c51-e62be1fabddc" containerID="84c825877221a22b36b1914b79d7616d6db6ea34cd96c6bd4a7c56332b2e099a" exitCode=0
Jul 07 14:40:18.452680 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:18.452259    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d6996576b-dg4bw"
Jul 07 14:40:18.452680 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:18.452303    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d6996576b-dg4bw" event={"ID":"ad48c566-1ca3-4467-8c51-e62be1fabddc","Type":"ContainerDied","Data":"84c825877221a22b36b1914b79d7616d6db6ea34cd96c6bd4a7c56332b2e099a"}
Jul 07 14:40:18.452680 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:18.452345    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d6996576b-dg4bw" event={"ID":"ad48c566-1ca3-4467-8c51-e62be1fabddc","Type":"ContainerDied","Data":"e3d552b8ca926bc8cc8a55849b4f8d4ace68c2048a1d681759b9ea6f4479c80a"}
Jul 07 14:40:18.452680 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:18.452362    2583 scope.go:117] "RemoveContainer" containerID="84c825877221a22b36b1914b79d7616d6db6ea34cd96c6bd4a7c56332b2e099a"
Jul 07 14:40:18.461117 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:18.461089    2583 scope.go:117] "RemoveContainer" containerID="84c825877221a22b36b1914b79d7616d6db6ea34cd96c6bd4a7c56332b2e099a"
Jul 07 14:40:18.461373 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:40:18.461349    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84c825877221a22b36b1914b79d7616d6db6ea34cd96c6bd4a7c56332b2e099a\": container with ID starting with 84c825877221a22b36b1914b79d7616d6db6ea34cd96c6bd4a7c56332b2e099a not found: ID does not exist" containerID="84c825877221a22b36b1914b79d7616d6db6ea34cd96c6bd4a7c56332b2e099a"
Jul 07 14:40:18.461500 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:18.461379    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84c825877221a22b36b1914b79d7616d6db6ea34cd96c6bd4a7c56332b2e099a"} err="failed to get container status \"84c825877221a22b36b1914b79d7616d6db6ea34cd96c6bd4a7c56332b2e099a\": rpc error: code = NotFound desc = could not find container \"84c825877221a22b36b1914b79d7616d6db6ea34cd96c6bd4a7c56332b2e099a\": container with ID starting with 84c825877221a22b36b1914b79d7616d6db6ea34cd96c6bd4a7c56332b2e099a not found: ID does not exist"
Jul 07 14:40:18.466144 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:18.466116    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d6996576b-dg4bw"]
Jul 07 14:40:18.471173 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:18.471145    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d6996576b-dg4bw"]
Jul 07 14:40:20.099828 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:20.099793    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad48c566-1ca3-4467-8c51-e62be1fabddc" path="/var/lib/kubelet/pods/ad48c566-1ca3-4467-8c51-e62be1fabddc/volumes"
Jul 07 14:40:37.846781 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:37.846740    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:40:47.836587 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:47.836556    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:40:59.439113 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:40:59.439079    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:41:08.332175 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:41:08.332140    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:41:18.533918 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:41:18.533832    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:41:26.019753 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:41:26.019720    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:41:26.022805 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:41:26.022784    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:41:45.446188 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:41:45.446154    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:41:56.533945 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:41:56.533913    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:42:32.837529 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:42:32.837492    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:42:42.236504 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:42:42.236474    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:42:50.635253 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:42:50.635167    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:43:00.732360 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:43:00.732319    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:43:10.538860 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:43:10.538824    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:43:19.343667 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:43:19.343619    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:43:31.529792 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:43:31.529749    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:43:40.435578 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:43:40.435543    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:44:03.935578 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:44:03.935539    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:44:16.142594 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:44:16.142552    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:44:40.239755 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:44:40.239713    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:44:51.532875 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:44:51.532842    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:45:05.942527 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:45:05.942488    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:45:16.746599 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:45:16.746565    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:45:25.942890 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:45:25.942854    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:45:36.538596 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:45:36.538557    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:45:44.841296 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:45:44.841258    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:46:26.050686 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:46:26.050645    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:46:26.052824 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:46:26.052800    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:47:35.634271 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:47:35.634233    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:47:44.435627 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:47:44.435590    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:47:57.138766 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:47:57.138731    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:48:08.042547 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:48:08.042504    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:48:20.843647 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:48:20.843610    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:48:29.243085 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:48:29.243053    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:48:39.149024 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:48:39.148987    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:48:49.237623 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:48:49.237542    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:48:59.852015 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:48:59.851966    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:49:10.639859 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:49:10.639819    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:49:22.038391 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:49:22.038352    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:49:44.778903 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:49:44.778864    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:49:53.355641 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:49:53.355605    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:50:02.562309 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:50:02.562275    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:50:11.742099 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:50:11.742061    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:50:29.256409 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:50:29.256324    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:50:40.342797 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:50:40.342764    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:50:49.437906 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:50:49.437863    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:50:59.241249 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:50:59.241213    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:51:07.450381 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:51:07.450335    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:51:18.640021 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:51:18.639966    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:51:26.076235 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:51:26.076200    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:51:26.076235 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:51:26.076217    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:51:32.370040 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:51:32.370008    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:51:41.556186 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:51:41.556153    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:51:50.991209 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:51:50.991117    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:52:02.853517 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:52:02.853459    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:52:15.531370 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:52:15.531335    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:52:24.345072 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:52:24.345041    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:52:36.444295 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:52:36.444256    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:52:48.740195 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:52:48.740152    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:53:01.733456 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:53:01.733420    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:53:10.144275 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:53:10.144238    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:53:20.060255 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:53:20.060166    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:53:27.434866 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:53:27.434826    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:53:36.541789 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:53:36.541755    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:53:55.335769 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:53:55.335726    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:54:06.745885 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:54:06.745851    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:54:15.740370 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:54:15.740328    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:54:27.843954 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:54:27.843912    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:55:47.635920 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:55:47.635883    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:55:54.741472 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:55:54.741439    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:56:03.331899 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:03.331869    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:56:14.459013 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:14.458966    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:56:22.552012 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:22.551909    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:56:26.107813 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:26.107776    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:56:26.108951 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:26.108932    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 14:56:52.270218 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:52.270182    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:56:55.545136 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.545104    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"]
Jul 07 14:56:55.545528 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.545432    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad48c566-1ca3-4467-8c51-e62be1fabddc" containerName="authorino"
Jul 07 14:56:55.545528 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.545450    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad48c566-1ca3-4467-8c51-e62be1fabddc" containerName="authorino"
Jul 07 14:56:55.545600 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.545585    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad48c566-1ca3-4467-8c51-e62be1fabddc" containerName="authorino"
Jul 07 14:56:55.548674 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.548660    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.551199 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.551175    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4afb43ce\"/\"kube-root-ca.crt\""
Jul 07 14:56:55.551199 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.551189    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4afb43ce\"/\"openshift-service-ca.crt\""
Jul 07 14:56:55.551385 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.551189    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4afb43ce\"/\"default-dockercfg-9f778\""
Jul 07 14:56:55.551729 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.551712    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4afb43ce\"/\"e2e-lifecycle-model-4afb43ce-kserve-self-signed-certs\""
Jul 07 14:56:55.559060 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.559036    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"]
Jul 07 14:56:55.704114 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.704082    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-dshm\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.704114 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.704123    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmwpk\" (UniqueName: \"kubernetes.io/projected/ad2ccccc-2073-409e-ac5d-7676d964b9a9-kube-api-access-rmwpk\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.704359 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.704148    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-model-cache\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.704359 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.704219    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-home\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.704359 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.704235    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad2ccccc-2073-409e-ac5d-7676d964b9a9-tls-certs\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.704359 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.704251    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-tmp-dir\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.804947 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.804843    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmwpk\" (UniqueName: \"kubernetes.io/projected/ad2ccccc-2073-409e-ac5d-7676d964b9a9-kube-api-access-rmwpk\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.804947 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.804905    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-model-cache\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.805219 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.805015    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-home\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.805219 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.805040    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad2ccccc-2073-409e-ac5d-7676d964b9a9-tls-certs\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.805219 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.805065    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-tmp-dir\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.805219 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.805125    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-dshm\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.805521 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.805488    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-model-cache\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.805634 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.805550    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-tmp-dir\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.805634 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.805578    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-home\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.807504 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.807481    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-dshm\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.807711 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.807694    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad2ccccc-2073-409e-ac5d-7676d964b9a9-tls-certs\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.816420 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.816392    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmwpk\" (UniqueName: \"kubernetes.io/projected/ad2ccccc-2073-409e-ac5d-7676d964b9a9-kube-api-access-rmwpk\") pod \"e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") " pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.858371 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.858327    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:56:55.999161 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:55.999137    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"]
Jul 07 14:56:56.001494 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:56:56.001468    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad2ccccc_2073_409e_ac5d_7676d964b9a9.slice/crio-1508564d32f84e0d20f4002f3b812124f7f44db303fd462a495b1034b2264b8e WatchSource:0}: Error finding container 1508564d32f84e0d20f4002f3b812124f7f44db303fd462a495b1034b2264b8e: Status 404 returned error can't find the container with id 1508564d32f84e0d20f4002f3b812124f7f44db303fd462a495b1034b2264b8e
Jul 07 14:56:56.643507 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:56.643471    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s" event={"ID":"ad2ccccc-2073-409e-ac5d-7676d964b9a9","Type":"ContainerStarted","Data":"ca640cdb44773063dc685a7f5195a5c962474a20d2de8938db02513d31e48e6f"}
Jul 07 14:56:56.643902 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:56.643517    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s" event={"ID":"ad2ccccc-2073-409e-ac5d-7676d964b9a9","Type":"ContainerStarted","Data":"1508564d32f84e0d20f4002f3b812124f7f44db303fd462a495b1034b2264b8e"}
Jul 07 14:56:56.666180 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:56:56.666130    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s" podStartSLOduration=1.666091677 podStartE2EDuration="1.666091677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:56:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:56:56.665503714 +0000 UTC m=+1831.128403656" watchObservedRunningTime="2026-07-07 14:56:56.666091677 +0000 UTC m=+1831.128991618"
Jul 07 14:57:05.859125 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:05.859076    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:57:05.859532 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:05.859143    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:57:05.871526 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:05.871499    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:57:06.698733 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:06.698702    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:57:06.748421 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:06.748388    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:57:17.878685 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:17.878649    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:57:25.088808 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:25.088768    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"]
Jul 07 14:57:30.046391 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:30.046359    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:57:40.109428 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.109375    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s" podUID="ad2ccccc-2073-409e-ac5d-7676d964b9a9" containerName="main" containerID="cri-o://ca640cdb44773063dc685a7f5195a5c962474a20d2de8938db02513d31e48e6f" gracePeriod=45
Jul 07 14:57:40.362630 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.362568    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:57:40.470316 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.470275    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-home\") pod \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") "
Jul 07 14:57:40.470525 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.470331    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-dshm\") pod \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") "
Jul 07 14:57:40.470525 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.470363    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmwpk\" (UniqueName: \"kubernetes.io/projected/ad2ccccc-2073-409e-ac5d-7676d964b9a9-kube-api-access-rmwpk\") pod \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") "
Jul 07 14:57:40.470525 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.470383    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-tmp-dir\") pod \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") "
Jul 07 14:57:40.470525 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.470408    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-model-cache\") pod \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") "
Jul 07 14:57:40.470525 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.470468    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad2ccccc-2073-409e-ac5d-7676d964b9a9-tls-certs\") pod \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\" (UID: \"ad2ccccc-2073-409e-ac5d-7676d964b9a9\") "
Jul 07 14:57:40.470769 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.470630    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-home" (OuterVolumeSpecName: "home") pod "ad2ccccc-2073-409e-ac5d-7676d964b9a9" (UID: "ad2ccccc-2073-409e-ac5d-7676d964b9a9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:57:40.470769 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.470674    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ad2ccccc-2073-409e-ac5d-7676d964b9a9" (UID: "ad2ccccc-2073-409e-ac5d-7676d964b9a9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:57:40.470769 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.470716    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-model-cache" (OuterVolumeSpecName: "model-cache") pod "ad2ccccc-2073-409e-ac5d-7676d964b9a9" (UID: "ad2ccccc-2073-409e-ac5d-7676d964b9a9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:57:40.470906 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.470790    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-home\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:57:40.470906 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.470807    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-tmp-dir\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:57:40.470906 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.470820    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-model-cache\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:57:40.472799 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.472765    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad2ccccc-2073-409e-ac5d-7676d964b9a9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ad2ccccc-2073-409e-ac5d-7676d964b9a9" (UID: "ad2ccccc-2073-409e-ac5d-7676d964b9a9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 14:57:40.472799 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.472788    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad2ccccc-2073-409e-ac5d-7676d964b9a9-kube-api-access-rmwpk" (OuterVolumeSpecName: "kube-api-access-rmwpk") pod "ad2ccccc-2073-409e-ac5d-7676d964b9a9" (UID: "ad2ccccc-2073-409e-ac5d-7676d964b9a9"). InnerVolumeSpecName "kube-api-access-rmwpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 14:57:40.472962 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.472808    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-dshm" (OuterVolumeSpecName: "dshm") pod "ad2ccccc-2073-409e-ac5d-7676d964b9a9" (UID: "ad2ccccc-2073-409e-ac5d-7676d964b9a9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 14:57:40.572129 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.572097    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad2ccccc-2073-409e-ac5d-7676d964b9a9-dshm\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:57:40.572129 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.572122    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmwpk\" (UniqueName: \"kubernetes.io/projected/ad2ccccc-2073-409e-ac5d-7676d964b9a9-kube-api-access-rmwpk\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:57:40.572129 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.572134    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad2ccccc-2073-409e-ac5d-7676d964b9a9-tls-certs\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 14:57:40.835765 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.835721    2583 generic.go:356] "Generic (PLEG): container finished" podID="ad2ccccc-2073-409e-ac5d-7676d964b9a9" containerID="ca640cdb44773063dc685a7f5195a5c962474a20d2de8938db02513d31e48e6f" exitCode=0
Jul 07 14:57:40.835765 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.835767    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"
Jul 07 14:57:40.835994 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.835778    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s" event={"ID":"ad2ccccc-2073-409e-ac5d-7676d964b9a9","Type":"ContainerDied","Data":"ca640cdb44773063dc685a7f5195a5c962474a20d2de8938db02513d31e48e6f"}
Jul 07 14:57:40.835994 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.835809    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s" event={"ID":"ad2ccccc-2073-409e-ac5d-7676d964b9a9","Type":"ContainerDied","Data":"1508564d32f84e0d20f4002f3b812124f7f44db303fd462a495b1034b2264b8e"}
Jul 07 14:57:40.835994 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.835825    2583 scope.go:117] "RemoveContainer" containerID="ca640cdb44773063dc685a7f5195a5c962474a20d2de8938db02513d31e48e6f"
Jul 07 14:57:40.844812 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.844790    2583 scope.go:117] "RemoveContainer" containerID="ca640cdb44773063dc685a7f5195a5c962474a20d2de8938db02513d31e48e6f"
Jul 07 14:57:40.845115 ip-10-0-129-1 kubenswrapper[2583]: E0707 14:57:40.845096    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca640cdb44773063dc685a7f5195a5c962474a20d2de8938db02513d31e48e6f\": container with ID starting with ca640cdb44773063dc685a7f5195a5c962474a20d2de8938db02513d31e48e6f not found: ID does not exist" containerID="ca640cdb44773063dc685a7f5195a5c962474a20d2de8938db02513d31e48e6f"
Jul 07 14:57:40.845166 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.845125    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca640cdb44773063dc685a7f5195a5c962474a20d2de8938db02513d31e48e6f"} err="failed to get container status \"ca640cdb44773063dc685a7f5195a5c962474a20d2de8938db02513d31e48e6f\": rpc error: code = NotFound desc = could not find container \"ca640cdb44773063dc685a7f5195a5c962474a20d2de8938db02513d31e48e6f\": container with ID starting with ca640cdb44773063dc685a7f5195a5c962474a20d2de8938db02513d31e48e6f not found: ID does not exist"
Jul 07 14:57:40.858636 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.858608    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"]
Jul 07 14:57:40.866158 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:40.866131    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-4afb43ce/e2e-lifecycle-model-4afb43ce-kserve-f4d6dbd95-pqq4s"]
Jul 07 14:57:42.101309 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:42.101268    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad2ccccc-2073-409e-ac5d-7676d964b9a9" path="/var/lib/kubelet/pods/ad2ccccc-2073-409e-ac5d-7676d964b9a9/volumes"
Jul 07 14:57:50.172133 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:57:50.172049    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:58:04.847942 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:58:04.847896    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:58:14.346680 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:58:14.346647    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:59:03.033390 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:03.033355    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:59:16.944795 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:16.944757    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:59:21.669064 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.668988    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"]
Jul 07 14:59:21.669426 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.669309    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad2ccccc-2073-409e-ac5d-7676d964b9a9" containerName="main"
Jul 07 14:59:21.669426 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.669320    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad2ccccc-2073-409e-ac5d-7676d964b9a9" containerName="main"
Jul 07 14:59:21.669426 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.669375    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad2ccccc-2073-409e-ac5d-7676d964b9a9" containerName="main"
Jul 07 14:59:21.672073 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.672055    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.674212 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.674190    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-17aa9c\"/\"kube-root-ca.crt\""
Jul 07 14:59:21.675413 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.675015    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-17aa9c\"/\"default-dockercfg-nb7pc\""
Jul 07 14:59:21.675682 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.675659    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-17aa9c\"/\"test-model-17aa9c-kserve-self-signed-certs\""
Jul 07 14:59:21.675903 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.675878    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-17aa9c\"/\"openshift-service-ca.crt\""
Jul 07 14:59:21.686528 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.686500    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"]
Jul 07 14:59:21.765478 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.765434    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-model-cache\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.765653 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.765513    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/812a1f72-0523-4903-9202-1e17ef73cec0-tls-certs\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.765653 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.765547    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-home\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.765653 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.765579    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-dshm\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.765653 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.765634    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzrn2\" (UniqueName: \"kubernetes.io/projected/812a1f72-0523-4903-9202-1e17ef73cec0-kube-api-access-fzrn2\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.765789 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.765690    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-tmp-dir\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.866740 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.866700    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzrn2\" (UniqueName: \"kubernetes.io/projected/812a1f72-0523-4903-9202-1e17ef73cec0-kube-api-access-fzrn2\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.866937 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.866750    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-tmp-dir\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.866937 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.866776    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-model-cache\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.866937 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.866833    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/812a1f72-0523-4903-9202-1e17ef73cec0-tls-certs\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.866937 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.866860    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-home\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.867216 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.866965    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-dshm\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.867273 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.867236    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-tmp-dir\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.867324 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.867282    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-model-cache\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.867324 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.867301    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-home\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.869168 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.869151    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-dshm\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.869342 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.869325    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/812a1f72-0523-4903-9202-1e17ef73cec0-tls-certs\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.874990 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.874954    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzrn2\" (UniqueName: \"kubernetes.io/projected/812a1f72-0523-4903-9202-1e17ef73cec0-kube-api-access-fzrn2\") pod \"test-model-17aa9c-kserve-b4db96989-jcvsg\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") " pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:21.984453 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:21.984413    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:22.127362 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:22.127335    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"]
Jul 07 14:59:22.129215 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:59:22.129190    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod812a1f72_0523_4903_9202_1e17ef73cec0.slice/crio-7681ef300f80edfa208b44cdc316b0b63219d64da2c66e1e9a441dbf865a3c95 WatchSource:0}: Error finding container 7681ef300f80edfa208b44cdc316b0b63219d64da2c66e1e9a441dbf865a3c95: Status 404 returned error can't find the container with id 7681ef300f80edfa208b44cdc316b0b63219d64da2c66e1e9a441dbf865a3c95
Jul 07 14:59:22.279172 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:22.279082    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg" event={"ID":"812a1f72-0523-4903-9202-1e17ef73cec0","Type":"ContainerStarted","Data":"d3bdbbb00789e4dd40d45b0263177e0cab86cca46813d2627b942c40f5e76d53"}
Jul 07 14:59:22.279172 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:22.279128    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg" event={"ID":"812a1f72-0523-4903-9202-1e17ef73cec0","Type":"ContainerStarted","Data":"7681ef300f80edfa208b44cdc316b0b63219d64da2c66e1e9a441dbf865a3c95"}
Jul 07 14:59:27.443450 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:27.443376    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg" podStartSLOduration=6.443351366 podStartE2EDuration="6.443351366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:59:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:59:22.304118782 +0000 UTC m=+1976.767018725" watchObservedRunningTime="2026-07-07 14:59:27.443351366 +0000 UTC m=+1981.906251396"
Jul 07 14:59:27.443962 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:27.443880    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:59:31.984824 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:31.984785    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:31.984824 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:31.984836    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:31.997490 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:31.997461    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:32.335464 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:32.335381    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 14:59:41.634440 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.634403    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"]
Jul 07 14:59:41.637623 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.637604    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.640278 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.640234    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-54f2c7\"/\"test-model-54f2c7-kserve-self-signed-certs\""
Jul 07 14:59:41.640402 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.640352    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-54f2c7\"/\"openshift-service-ca.crt\""
Jul 07 14:59:41.640402 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.640237    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-54f2c7\"/\"kube-root-ca.crt\""
Jul 07 14:59:41.640521 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.640409    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-54f2c7\"/\"default-dockercfg-6sf9j\""
Jul 07 14:59:41.648657 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.648631    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"]
Jul 07 14:59:41.738386 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.738347    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-home\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.738577 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.738406    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-dshm\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.738577 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.738437    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfbmw\" (UniqueName: \"kubernetes.io/projected/aec53b37-aca9-48a6-8401-11b18ffea08b-kube-api-access-xfbmw\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.738577 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.738470    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aec53b37-aca9-48a6-8401-11b18ffea08b-tls-certs\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.738577 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.738495    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-model-cache\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.738577 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.738532    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-tmp-dir\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.839711 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.839670    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aec53b37-aca9-48a6-8401-11b18ffea08b-tls-certs\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.839711 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.839711    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-model-cache\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.839934 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.839729    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-tmp-dir\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.839934 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.839785    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-home\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.839934 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.839819    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-dshm\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.839934 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.839836    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfbmw\" (UniqueName: \"kubernetes.io/projected/aec53b37-aca9-48a6-8401-11b18ffea08b-kube-api-access-xfbmw\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.840222 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.840201    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-tmp-dir\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.840324 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.840260    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-model-cache\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.840324 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.840299    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-home\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.842078 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.842061    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-dshm\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.842278 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.842259    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aec53b37-aca9-48a6-8401-11b18ffea08b-tls-certs\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.847964 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.847941    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfbmw\" (UniqueName: \"kubernetes.io/projected/aec53b37-aca9-48a6-8401-11b18ffea08b-kube-api-access-xfbmw\") pod \"test-model-54f2c7-kserve-5d7dfffcb7-nn76x\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") " pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:41.947178 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:41.947142    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:42.086633 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:42.086604    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"]
Jul 07 14:59:42.089592 ip-10-0-129-1 kubenswrapper[2583]: W0707 14:59:42.089563    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaec53b37_aca9_48a6_8401_11b18ffea08b.slice/crio-332433516edbed463017d24436fd4317390e7f35cffd6b6b1e22c1f5dfb844a6 WatchSource:0}: Error finding container 332433516edbed463017d24436fd4317390e7f35cffd6b6b1e22c1f5dfb844a6: Status 404 returned error can't find the container with id 332433516edbed463017d24436fd4317390e7f35cffd6b6b1e22c1f5dfb844a6
Jul 07 14:59:42.372548 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:42.372462    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x" event={"ID":"aec53b37-aca9-48a6-8401-11b18ffea08b","Type":"ContainerStarted","Data":"d5a0d4399294e2979585b6080e1dd1e10cbb3aeb758fc29345075c52ece587b4"}
Jul 07 14:59:42.372548 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:42.372503    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x" event={"ID":"aec53b37-aca9-48a6-8401-11b18ffea08b","Type":"ContainerStarted","Data":"332433516edbed463017d24436fd4317390e7f35cffd6b6b1e22c1f5dfb844a6"}
Jul 07 14:59:42.390851 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:42.390792    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x" podStartSLOduration=1.3907755960000001 podStartE2EDuration="1.390775596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 14:59:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 14:59:42.389138824 +0000 UTC m=+1996.852038779" watchObservedRunningTime="2026-07-07 14:59:42.390775596 +0000 UTC m=+1996.853675570"
Jul 07 14:59:45.533129 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:45.533095    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 14:59:51.947433 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:51.947392    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:51.948060 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:51.947454    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:51.960294 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:51.960264    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:52.426873 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:52.426788    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 14:59:58.242283 ip-10-0-129-1 kubenswrapper[2583]: I0707 14:59:58.242251    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:00:06.619425 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:06.619394    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"]
Jul 07 15:00:07.625942 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:07.625906    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"]
Jul 07 15:00:10.533551 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:10.533518    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:00:21.639778 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.639719    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg" podUID="812a1f72-0523-4903-9202-1e17ef73cec0" containerName="main" containerID="cri-o://d3bdbbb00789e4dd40d45b0263177e0cab86cca46813d2627b942c40f5e76d53" gracePeriod=45
Jul 07 15:00:21.891507 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.891445    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 15:00:21.978689 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.978654    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-home\") pod \"812a1f72-0523-4903-9202-1e17ef73cec0\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") "
Jul 07 15:00:21.978689 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.978695    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-tmp-dir\") pod \"812a1f72-0523-4903-9202-1e17ef73cec0\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") "
Jul 07 15:00:21.978948 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.978769    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-model-cache\") pod \"812a1f72-0523-4903-9202-1e17ef73cec0\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") "
Jul 07 15:00:21.978948 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.978807    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/812a1f72-0523-4903-9202-1e17ef73cec0-tls-certs\") pod \"812a1f72-0523-4903-9202-1e17ef73cec0\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") "
Jul 07 15:00:21.978948 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.978831    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzrn2\" (UniqueName: \"kubernetes.io/projected/812a1f72-0523-4903-9202-1e17ef73cec0-kube-api-access-fzrn2\") pod \"812a1f72-0523-4903-9202-1e17ef73cec0\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") "
Jul 07 15:00:21.978948 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.978858    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-dshm\") pod \"812a1f72-0523-4903-9202-1e17ef73cec0\" (UID: \"812a1f72-0523-4903-9202-1e17ef73cec0\") "
Jul 07 15:00:21.978948 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.978891    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-home" (OuterVolumeSpecName: "home") pod "812a1f72-0523-4903-9202-1e17ef73cec0" (UID: "812a1f72-0523-4903-9202-1e17ef73cec0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:00:21.979180 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.979008    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "812a1f72-0523-4903-9202-1e17ef73cec0" (UID: "812a1f72-0523-4903-9202-1e17ef73cec0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:00:21.979180 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.979061    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-model-cache" (OuterVolumeSpecName: "model-cache") pod "812a1f72-0523-4903-9202-1e17ef73cec0" (UID: "812a1f72-0523-4903-9202-1e17ef73cec0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:00:21.979180 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.979170    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-home\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:00:21.979276 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.979188    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-tmp-dir\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:00:21.979276 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.979201    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-model-cache\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:00:21.981164 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.981131    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/812a1f72-0523-4903-9202-1e17ef73cec0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "812a1f72-0523-4903-9202-1e17ef73cec0" (UID: "812a1f72-0523-4903-9202-1e17ef73cec0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:00:21.981164 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.981146    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-dshm" (OuterVolumeSpecName: "dshm") pod "812a1f72-0523-4903-9202-1e17ef73cec0" (UID: "812a1f72-0523-4903-9202-1e17ef73cec0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:00:21.981343 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:21.981227    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/812a1f72-0523-4903-9202-1e17ef73cec0-kube-api-access-fzrn2" (OuterVolumeSpecName: "kube-api-access-fzrn2") pod "812a1f72-0523-4903-9202-1e17ef73cec0" (UID: "812a1f72-0523-4903-9202-1e17ef73cec0"). InnerVolumeSpecName "kube-api-access-fzrn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:00:22.079928 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.079887    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/812a1f72-0523-4903-9202-1e17ef73cec0-tls-certs\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:00:22.079928 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.079916    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzrn2\" (UniqueName: \"kubernetes.io/projected/812a1f72-0523-4903-9202-1e17ef73cec0-kube-api-access-fzrn2\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:00:22.079928 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.079927    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/812a1f72-0523-4903-9202-1e17ef73cec0-dshm\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:00:22.543476 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.543437    2583 generic.go:356] "Generic (PLEG): container finished" podID="812a1f72-0523-4903-9202-1e17ef73cec0" containerID="d3bdbbb00789e4dd40d45b0263177e0cab86cca46813d2627b942c40f5e76d53" exitCode=0
Jul 07 15:00:22.543633 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.543494    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"
Jul 07 15:00:22.543633 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.543513    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg" event={"ID":"812a1f72-0523-4903-9202-1e17ef73cec0","Type":"ContainerDied","Data":"d3bdbbb00789e4dd40d45b0263177e0cab86cca46813d2627b942c40f5e76d53"}
Jul 07 15:00:22.543633 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.543558    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg" event={"ID":"812a1f72-0523-4903-9202-1e17ef73cec0","Type":"ContainerDied","Data":"7681ef300f80edfa208b44cdc316b0b63219d64da2c66e1e9a441dbf865a3c95"}
Jul 07 15:00:22.543633 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.543579    2583 scope.go:117] "RemoveContainer" containerID="d3bdbbb00789e4dd40d45b0263177e0cab86cca46813d2627b942c40f5e76d53"
Jul 07 15:00:22.552672 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.552655    2583 scope.go:117] "RemoveContainer" containerID="d3bdbbb00789e4dd40d45b0263177e0cab86cca46813d2627b942c40f5e76d53"
Jul 07 15:00:22.552986 ip-10-0-129-1 kubenswrapper[2583]: E0707 15:00:22.552956    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3bdbbb00789e4dd40d45b0263177e0cab86cca46813d2627b942c40f5e76d53\": container with ID starting with d3bdbbb00789e4dd40d45b0263177e0cab86cca46813d2627b942c40f5e76d53 not found: ID does not exist" containerID="d3bdbbb00789e4dd40d45b0263177e0cab86cca46813d2627b942c40f5e76d53"
Jul 07 15:00:22.553035 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.553006    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3bdbbb00789e4dd40d45b0263177e0cab86cca46813d2627b942c40f5e76d53"} err="failed to get container status \"d3bdbbb00789e4dd40d45b0263177e0cab86cca46813d2627b942c40f5e76d53\": rpc error: code = NotFound desc = could not find container \"d3bdbbb00789e4dd40d45b0263177e0cab86cca46813d2627b942c40f5e76d53\": container with ID starting with d3bdbbb00789e4dd40d45b0263177e0cab86cca46813d2627b942c40f5e76d53 not found: ID does not exist"
Jul 07 15:00:22.558947 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.558921    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"]
Jul 07 15:00:22.562192 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.562169    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-17aa9c/test-model-17aa9c-kserve-b4db96989-jcvsg"]
Jul 07 15:00:22.645780 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.645743    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x" podUID="aec53b37-aca9-48a6-8401-11b18ffea08b" containerName="main" containerID="cri-o://d5a0d4399294e2979585b6080e1dd1e10cbb3aeb758fc29345075c52ece587b4" gracePeriod=45
Jul 07 15:00:22.900272 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.900246    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 15:00:22.987521 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.987481    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-dshm\") pod \"aec53b37-aca9-48a6-8401-11b18ffea08b\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") "
Jul 07 15:00:22.987712 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.987546    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aec53b37-aca9-48a6-8401-11b18ffea08b-tls-certs\") pod \"aec53b37-aca9-48a6-8401-11b18ffea08b\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") "
Jul 07 15:00:22.987712 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.987596    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-home\") pod \"aec53b37-aca9-48a6-8401-11b18ffea08b\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") "
Jul 07 15:00:22.987712 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.987628    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-model-cache\") pod \"aec53b37-aca9-48a6-8401-11b18ffea08b\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") "
Jul 07 15:00:22.987712 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.987663    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfbmw\" (UniqueName: \"kubernetes.io/projected/aec53b37-aca9-48a6-8401-11b18ffea08b-kube-api-access-xfbmw\") pod \"aec53b37-aca9-48a6-8401-11b18ffea08b\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") "
Jul 07 15:00:22.987712 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.987695    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-tmp-dir\") pod \"aec53b37-aca9-48a6-8401-11b18ffea08b\" (UID: \"aec53b37-aca9-48a6-8401-11b18ffea08b\") "
Jul 07 15:00:22.988002 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.987892    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-home" (OuterVolumeSpecName: "home") pod "aec53b37-aca9-48a6-8401-11b18ffea08b" (UID: "aec53b37-aca9-48a6-8401-11b18ffea08b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:00:22.988002 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.987909    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-model-cache" (OuterVolumeSpecName: "model-cache") pod "aec53b37-aca9-48a6-8401-11b18ffea08b" (UID: "aec53b37-aca9-48a6-8401-11b18ffea08b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:00:22.988002 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.987967    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aec53b37-aca9-48a6-8401-11b18ffea08b" (UID: "aec53b37-aca9-48a6-8401-11b18ffea08b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:00:22.988163 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.988068    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-home\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:00:22.989943 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.989919    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aec53b37-aca9-48a6-8401-11b18ffea08b-kube-api-access-xfbmw" (OuterVolumeSpecName: "kube-api-access-xfbmw") pod "aec53b37-aca9-48a6-8401-11b18ffea08b" (UID: "aec53b37-aca9-48a6-8401-11b18ffea08b"). InnerVolumeSpecName "kube-api-access-xfbmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:00:22.990053 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.989920    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-dshm" (OuterVolumeSpecName: "dshm") pod "aec53b37-aca9-48a6-8401-11b18ffea08b" (UID: "aec53b37-aca9-48a6-8401-11b18ffea08b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:00:22.990053 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:22.989920    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aec53b37-aca9-48a6-8401-11b18ffea08b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aec53b37-aca9-48a6-8401-11b18ffea08b" (UID: "aec53b37-aca9-48a6-8401-11b18ffea08b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:00:23.090720 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:23.090632    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-dshm\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:00:23.090720 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:23.090680    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aec53b37-aca9-48a6-8401-11b18ffea08b-tls-certs\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:00:23.090720 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:23.090697    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-model-cache\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:00:23.090720 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:23.090706    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfbmw\" (UniqueName: \"kubernetes.io/projected/aec53b37-aca9-48a6-8401-11b18ffea08b-kube-api-access-xfbmw\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:00:23.090720 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:23.090715    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aec53b37-aca9-48a6-8401-11b18ffea08b-tmp-dir\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:00:23.549211 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:23.549170    2583 generic.go:356] "Generic (PLEG): container finished" podID="aec53b37-aca9-48a6-8401-11b18ffea08b" containerID="d5a0d4399294e2979585b6080e1dd1e10cbb3aeb758fc29345075c52ece587b4" exitCode=0
Jul 07 15:00:23.549423 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:23.549236    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"
Jul 07 15:00:23.549423 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:23.549264    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x" event={"ID":"aec53b37-aca9-48a6-8401-11b18ffea08b","Type":"ContainerDied","Data":"d5a0d4399294e2979585b6080e1dd1e10cbb3aeb758fc29345075c52ece587b4"}
Jul 07 15:00:23.549423 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:23.549316    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x" event={"ID":"aec53b37-aca9-48a6-8401-11b18ffea08b","Type":"ContainerDied","Data":"332433516edbed463017d24436fd4317390e7f35cffd6b6b1e22c1f5dfb844a6"}
Jul 07 15:00:23.549423 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:23.549355    2583 scope.go:117] "RemoveContainer" containerID="d5a0d4399294e2979585b6080e1dd1e10cbb3aeb758fc29345075c52ece587b4"
Jul 07 15:00:23.557946 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:23.557928    2583 scope.go:117] "RemoveContainer" containerID="d5a0d4399294e2979585b6080e1dd1e10cbb3aeb758fc29345075c52ece587b4"
Jul 07 15:00:23.558426 ip-10-0-129-1 kubenswrapper[2583]: E0707 15:00:23.558405    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5a0d4399294e2979585b6080e1dd1e10cbb3aeb758fc29345075c52ece587b4\": container with ID starting with d5a0d4399294e2979585b6080e1dd1e10cbb3aeb758fc29345075c52ece587b4 not found: ID does not exist" containerID="d5a0d4399294e2979585b6080e1dd1e10cbb3aeb758fc29345075c52ece587b4"
Jul 07 15:00:23.558492 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:23.558436    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a0d4399294e2979585b6080e1dd1e10cbb3aeb758fc29345075c52ece587b4"} err="failed to get container status \"d5a0d4399294e2979585b6080e1dd1e10cbb3aeb758fc29345075c52ece587b4\": rpc error: code = NotFound desc = could not find container \"d5a0d4399294e2979585b6080e1dd1e10cbb3aeb758fc29345075c52ece587b4\": container with ID starting with d5a0d4399294e2979585b6080e1dd1e10cbb3aeb758fc29345075c52ece587b4 not found: ID does not exist"
Jul 07 15:00:23.569193 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:23.569159    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"]
Jul 07 15:00:23.573428 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:23.573395    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-54f2c7/test-model-54f2c7-kserve-5d7dfffcb7-nn76x"]
Jul 07 15:00:24.101596 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:24.101550    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="812a1f72-0523-4903-9202-1e17ef73cec0" path="/var/lib/kubelet/pods/812a1f72-0523-4903-9202-1e17ef73cec0/volumes"
Jul 07 15:00:24.102129 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:24.102093    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aec53b37-aca9-48a6-8401-11b18ffea08b" path="/var/lib/kubelet/pods/aec53b37-aca9-48a6-8401-11b18ffea08b/volumes"
Jul 07 15:00:24.340346 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:24.340311    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:00:54.540970 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:00:54.540884    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:01:06.444743 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:01:06.444702    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:01:22.745108 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:01:22.745069    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:01:26.134562 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:01:26.134533    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 15:01:26.136071 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:01:26.136045    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 15:01:34.045052 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:01:34.045012    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:01:43.634916 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:01:43.634878    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:01:58.793517 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:01:58.793483    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:02:07.141052 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.141018    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"]
Jul 07 15:02:07.141533 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.141363    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="812a1f72-0523-4903-9202-1e17ef73cec0" containerName="main"
Jul 07 15:02:07.141533 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.141380    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="812a1f72-0523-4903-9202-1e17ef73cec0" containerName="main"
Jul 07 15:02:07.141533 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.141409    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aec53b37-aca9-48a6-8401-11b18ffea08b" containerName="main"
Jul 07 15:02:07.141533 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.141417    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="aec53b37-aca9-48a6-8401-11b18ffea08b" containerName="main"
Jul 07 15:02:07.141533 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.141488    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="aec53b37-aca9-48a6-8401-11b18ffea08b" containerName="main"
Jul 07 15:02:07.141533 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.141500    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="812a1f72-0523-4903-9202-1e17ef73cec0" containerName="main"
Jul 07 15:02:07.144511 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.144491    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.146701 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.146679    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-26fa43\"/\"kube-root-ca.crt\""
Jul 07 15:02:07.147465 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.147446    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-26fa43\"/\"auth-test-model-26fa43-kserve-self-signed-certs\""
Jul 07 15:02:07.147592 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.147477    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-26fa43\"/\"openshift-service-ca.crt\""
Jul 07 15:02:07.147656 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.147614    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-26fa43\"/\"default-dockercfg-ggs4z\""
Jul 07 15:02:07.156464 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.156436    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"]
Jul 07 15:02:07.291127 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.291089    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-model-cache\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.291127 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.291132    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-home\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.291364 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.291153    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2675923-2047-46b5-8337-65fd3a294e76-tls-certs\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.291364 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.291311    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd2pr\" (UniqueName: \"kubernetes.io/projected/d2675923-2047-46b5-8337-65fd3a294e76-kube-api-access-zd2pr\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.291364 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.291358    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-dshm\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.291469 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.291378    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-tmp-dir\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.391900 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.391812    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd2pr\" (UniqueName: \"kubernetes.io/projected/d2675923-2047-46b5-8337-65fd3a294e76-kube-api-access-zd2pr\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.391900 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.391858    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-dshm\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.392093 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.391969    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-tmp-dir\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.392093 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.392035    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-model-cache\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.392093 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.392067    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-home\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.392093 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.392083    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2675923-2047-46b5-8337-65fd3a294e76-tls-certs\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.392320 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.392301    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-tmp-dir\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.392424 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.392405    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-model-cache\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.392485 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.392465    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-home\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.394157 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.394133    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-dshm\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.394496 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.394476    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2675923-2047-46b5-8337-65fd3a294e76-tls-certs\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.400671 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.400648    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd2pr\" (UniqueName: \"kubernetes.io/projected/d2675923-2047-46b5-8337-65fd3a294e76-kube-api-access-zd2pr\") pod \"auth-test-model-26fa43-kserve-56c4974d8c-xmz4v\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") " pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.453663 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.453630    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:07.601992 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:07.601945    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"]
Jul 07 15:02:07.605130 ip-10-0-129-1 kubenswrapper[2583]: W0707 15:02:07.605102    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2675923_2047_46b5_8337_65fd3a294e76.slice/crio-05bc4ed4ec9d94407c2a2dd0bd905dac2f96844a9e60c00bd42fe9b2ef460b4c WatchSource:0}: Error finding container 05bc4ed4ec9d94407c2a2dd0bd905dac2f96844a9e60c00bd42fe9b2ef460b4c: Status 404 returned error can't find the container with id 05bc4ed4ec9d94407c2a2dd0bd905dac2f96844a9e60c00bd42fe9b2ef460b4c
Jul 07 15:02:08.000957 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:08.000924    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v" event={"ID":"d2675923-2047-46b5-8337-65fd3a294e76","Type":"ContainerStarted","Data":"2c22df1abf8d3cdfe7d2732dc75b039fc8c515840a741a4f8293fe8243e65489"}
Jul 07 15:02:08.000957 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:08.000961    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v" event={"ID":"d2675923-2047-46b5-8337-65fd3a294e76","Type":"ContainerStarted","Data":"05bc4ed4ec9d94407c2a2dd0bd905dac2f96844a9e60c00bd42fe9b2ef460b4c"}
Jul 07 15:02:08.023250 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:08.023198    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v" podStartSLOduration=1.023180623 podStartE2EDuration="1.023180623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:02:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:08.022990528 +0000 UTC m=+2142.485890466" watchObservedRunningTime="2026-07-07 15:02:08.023180623 +0000 UTC m=+2142.486080562"
Jul 07 15:02:13.940776 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:13.940736    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:02:17.454602 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:17.454552    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:17.455112 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:17.454620    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:17.467277 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:17.467246    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:18.059926 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:18.059894    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:02:29.318877 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.318829    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"]
Jul 07 15:02:29.323474 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.323451    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.326080 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.326051    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2f8e54\"/\"auth-test-model-2f8e54-kserve-self-signed-certs\""
Jul 07 15:02:29.326247 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.326061    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2f8e54\"/\"default-dockercfg-9vbt6\""
Jul 07 15:02:29.326247 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.326125    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2f8e54\"/\"kube-root-ca.crt\""
Jul 07 15:02:29.326247 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.326099    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2f8e54\"/\"openshift-service-ca.crt\""
Jul 07 15:02:29.332343 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.332310    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"]
Jul 07 15:02:29.374247 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.374206    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbec51e9-1b4b-4101-8c7e-1a61b3918361-tls-certs\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.374247 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.374253    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-dshm\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.374463 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.374285    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-model-cache\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.374463 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.374331    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmjcx\" (UniqueName: \"kubernetes.io/projected/dbec51e9-1b4b-4101-8c7e-1a61b3918361-kube-api-access-qmjcx\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.374463 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.374365    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-home\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.374463 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.374381    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-tmp-dir\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.475560 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.475521    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbec51e9-1b4b-4101-8c7e-1a61b3918361-tls-certs\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.475560 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.475566    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-dshm\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.475827 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.475598    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-model-cache\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.475827 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.475653    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmjcx\" (UniqueName: \"kubernetes.io/projected/dbec51e9-1b4b-4101-8c7e-1a61b3918361-kube-api-access-qmjcx\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.475827 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.475787    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-home\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.476012 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.475828    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-tmp-dir\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.476182 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.476160    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-model-cache\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.476276 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.476181    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-home\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.476276 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.476226    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-tmp-dir\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.477964 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.477940    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-dshm\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.478190 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.478175    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbec51e9-1b4b-4101-8c7e-1a61b3918361-tls-certs\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.483602 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.483577    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmjcx\" (UniqueName: \"kubernetes.io/projected/dbec51e9-1b4b-4101-8c7e-1a61b3918361-kube-api-access-qmjcx\") pod \"auth-test-model-2f8e54-kserve-67c86dc4f9-mf279\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") " pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.634645 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.634553    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:29.770609 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:29.770522    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"]
Jul 07 15:02:29.773286 ip-10-0-129-1 kubenswrapper[2583]: W0707 15:02:29.773252    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddbec51e9_1b4b_4101_8c7e_1a61b3918361.slice/crio-b79d7a389dc75d7c4bc0ed1d327f89a0313b5e2f967bac6d54b7660784726056 WatchSource:0}: Error finding container b79d7a389dc75d7c4bc0ed1d327f89a0313b5e2f967bac6d54b7660784726056: Status 404 returned error can't find the container with id b79d7a389dc75d7c4bc0ed1d327f89a0313b5e2f967bac6d54b7660784726056
Jul 07 15:02:30.100824 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:30.100794    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279" event={"ID":"dbec51e9-1b4b-4101-8c7e-1a61b3918361","Type":"ContainerStarted","Data":"75d2ac6d5f40eed4b4b658bd5994ddab3a598cc82f06fc4530bea2ec7fc077d4"}
Jul 07 15:02:30.100998 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:30.100832    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279" event={"ID":"dbec51e9-1b4b-4101-8c7e-1a61b3918361","Type":"ContainerStarted","Data":"b79d7a389dc75d7c4bc0ed1d327f89a0313b5e2f967bac6d54b7660784726056"}
Jul 07 15:02:30.118869 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:30.118820    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279" podStartSLOduration=1.118805354 podStartE2EDuration="1.118805354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:02:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:30.116895807 +0000 UTC m=+2164.579795748" watchObservedRunningTime="2026-07-07 15:02:30.118805354 +0000 UTC m=+2164.581705295"
Jul 07 15:02:32.549389 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:32.545404    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:02:39.635644 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:39.635608    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:39.636077 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:39.635878    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:39.648177 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:39.648150    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:40.151794 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:40.151684    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:02:57.142821 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:02:57.142783    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:03:14.336021 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:03:14.335988    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:03:27.756486 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:03:27.756435    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:03:44.950081 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:03:44.950042    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:03:59.344531 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:03:59.344450    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:04:13.340492 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:13.340459    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:04:25.567178 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:25.567140    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"]
Jul 07 15:04:26.540774 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:26.540743    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"]
Jul 07 15:04:29.744171 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:29.744137    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:04:37.486506 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.486468    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"]
Jul 07 15:04:37.489614 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.489593    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.491791 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.491771    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-26fa43\"/\"sub-test-model-26fa43-kserve-self-signed-certs\""
Jul 07 15:04:37.501457 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.501434    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"]
Jul 07 15:04:37.572416 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.572378    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-dshm\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.572416 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.572415    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnddz\" (UniqueName: \"kubernetes.io/projected/6b8bcc08-d711-4007-b532-a5516790b301-kube-api-access-mnddz\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.572655 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.572485    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-model-cache\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.572655 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.572517    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-tmp-dir\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.572655 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.572553    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-home\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.572655 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.572590    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8bcc08-d711-4007-b532-a5516790b301-tls-certs\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.673523 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.673469    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-dshm\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.673523 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.673520    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnddz\" (UniqueName: \"kubernetes.io/projected/6b8bcc08-d711-4007-b532-a5516790b301-kube-api-access-mnddz\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.673819 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.673546    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-model-cache\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.673819 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.673677    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-tmp-dir\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.673819 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.673740    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-home\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.673819 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.673799    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8bcc08-d711-4007-b532-a5516790b301-tls-certs\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.674025 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.673900    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-model-cache\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.674115 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.674092    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-tmp-dir\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.674151 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.674138    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-home\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.676069 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.676033    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-dshm\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.676354 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.676337    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8bcc08-d711-4007-b532-a5516790b301-tls-certs\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.683122 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.683088    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnddz\" (UniqueName: \"kubernetes.io/projected/6b8bcc08-d711-4007-b532-a5516790b301-kube-api-access-mnddz\") pod \"sub-test-model-26fa43-kserve-58594bcf67-gb8g9\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") " pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.798543 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.798460    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:37.937604 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:37.937577    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"]
Jul 07 15:04:37.939832 ip-10-0-129-1 kubenswrapper[2583]: W0707 15:04:37.939806    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b8bcc08_d711_4007_b532_a5516790b301.slice/crio-e93bafd05f6b4da9834a696ded48ff69b27e61a96ebb9f0b51e352d352c129a9 WatchSource:0}: Error finding container e93bafd05f6b4da9834a696ded48ff69b27e61a96ebb9f0b51e352d352c129a9: Status 404 returned error can't find the container with id e93bafd05f6b4da9834a696ded48ff69b27e61a96ebb9f0b51e352d352c129a9
Jul 07 15:04:38.653942 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:38.653907    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9" event={"ID":"6b8bcc08-d711-4007-b532-a5516790b301","Type":"ContainerStarted","Data":"e3de2dd1e6ceb699ed336cbfafd4063728c5f74fc53f82a74384a436694b706e"}
Jul 07 15:04:38.654329 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:38.653955    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9" event={"ID":"6b8bcc08-d711-4007-b532-a5516790b301","Type":"ContainerStarted","Data":"e93bafd05f6b4da9834a696ded48ff69b27e61a96ebb9f0b51e352d352c129a9"}
Jul 07 15:04:38.673692 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:38.673640    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9" podStartSLOduration=1.6736259329999998 podStartE2EDuration="1.673625933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:04:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:04:38.672438386 +0000 UTC m=+2293.135338330" watchObservedRunningTime="2026-07-07 15:04:38.673625933 +0000 UTC m=+2293.136525875"
Jul 07 15:04:40.587413 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:40.587369    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v" podUID="d2675923-2047-46b5-8337-65fd3a294e76" containerName="main" containerID="cri-o://2c22df1abf8d3cdfe7d2732dc75b039fc8c515840a741a4f8293fe8243e65489" gracePeriod=45
Jul 07 15:04:40.848739 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:40.848674    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:04:41.003084 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.003044    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd2pr\" (UniqueName: \"kubernetes.io/projected/d2675923-2047-46b5-8337-65fd3a294e76-kube-api-access-zd2pr\") pod \"d2675923-2047-46b5-8337-65fd3a294e76\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") "
Jul 07 15:04:41.003277 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.003131    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-model-cache\") pod \"d2675923-2047-46b5-8337-65fd3a294e76\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") "
Jul 07 15:04:41.003277 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.003161    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2675923-2047-46b5-8337-65fd3a294e76-tls-certs\") pod \"d2675923-2047-46b5-8337-65fd3a294e76\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") "
Jul 07 15:04:41.003277 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.003192    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-tmp-dir\") pod \"d2675923-2047-46b5-8337-65fd3a294e76\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") "
Jul 07 15:04:41.003277 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.003249    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-home\") pod \"d2675923-2047-46b5-8337-65fd3a294e76\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") "
Jul 07 15:04:41.003468 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.003282    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-dshm\") pod \"d2675923-2047-46b5-8337-65fd3a294e76\" (UID: \"d2675923-2047-46b5-8337-65fd3a294e76\") "
Jul 07 15:04:41.003468 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.003394    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-model-cache" (OuterVolumeSpecName: "model-cache") pod "d2675923-2047-46b5-8337-65fd3a294e76" (UID: "d2675923-2047-46b5-8337-65fd3a294e76"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:04:41.003580 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.003485    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-home" (OuterVolumeSpecName: "home") pod "d2675923-2047-46b5-8337-65fd3a294e76" (UID: "d2675923-2047-46b5-8337-65fd3a294e76"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:04:41.003580 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.003523    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d2675923-2047-46b5-8337-65fd3a294e76" (UID: "d2675923-2047-46b5-8337-65fd3a294e76"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:04:41.003669 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.003608    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-home\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:04:41.003669 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.003621    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-model-cache\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:04:41.003669 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.003631    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-tmp-dir\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:04:41.005511 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.005482    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-dshm" (OuterVolumeSpecName: "dshm") pod "d2675923-2047-46b5-8337-65fd3a294e76" (UID: "d2675923-2047-46b5-8337-65fd3a294e76"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:04:41.005770 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.005743    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2675923-2047-46b5-8337-65fd3a294e76-kube-api-access-zd2pr" (OuterVolumeSpecName: "kube-api-access-zd2pr") pod "d2675923-2047-46b5-8337-65fd3a294e76" (UID: "d2675923-2047-46b5-8337-65fd3a294e76"). InnerVolumeSpecName "kube-api-access-zd2pr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:04:41.005770 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.005743    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2675923-2047-46b5-8337-65fd3a294e76-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d2675923-2047-46b5-8337-65fd3a294e76" (UID: "d2675923-2047-46b5-8337-65fd3a294e76"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:04:41.104363 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.104280    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2675923-2047-46b5-8337-65fd3a294e76-dshm\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:04:41.104363 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.104311    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd2pr\" (UniqueName: \"kubernetes.io/projected/d2675923-2047-46b5-8337-65fd3a294e76-kube-api-access-zd2pr\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:04:41.104363 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.104322    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2675923-2047-46b5-8337-65fd3a294e76-tls-certs\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:04:41.560466 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.560418    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279" podUID="dbec51e9-1b4b-4101-8c7e-1a61b3918361" containerName="main" containerID="cri-o://75d2ac6d5f40eed4b4b658bd5994ddab3a598cc82f06fc4530bea2ec7fc077d4" gracePeriod=45
Jul 07 15:04:41.668418 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.668380    2583 generic.go:356] "Generic (PLEG): container finished" podID="d2675923-2047-46b5-8337-65fd3a294e76" containerID="2c22df1abf8d3cdfe7d2732dc75b039fc8c515840a741a4f8293fe8243e65489" exitCode=0
Jul 07 15:04:41.668418 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.668417    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"
Jul 07 15:04:41.668927 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.668439    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v" event={"ID":"d2675923-2047-46b5-8337-65fd3a294e76","Type":"ContainerDied","Data":"2c22df1abf8d3cdfe7d2732dc75b039fc8c515840a741a4f8293fe8243e65489"}
Jul 07 15:04:41.668927 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.668489    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v" event={"ID":"d2675923-2047-46b5-8337-65fd3a294e76","Type":"ContainerDied","Data":"05bc4ed4ec9d94407c2a2dd0bd905dac2f96844a9e60c00bd42fe9b2ef460b4c"}
Jul 07 15:04:41.668927 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.668507    2583 scope.go:117] "RemoveContainer" containerID="2c22df1abf8d3cdfe7d2732dc75b039fc8c515840a741a4f8293fe8243e65489"
Jul 07 15:04:41.676844 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.676827    2583 scope.go:117] "RemoveContainer" containerID="2c22df1abf8d3cdfe7d2732dc75b039fc8c515840a741a4f8293fe8243e65489"
Jul 07 15:04:41.677163 ip-10-0-129-1 kubenswrapper[2583]: E0707 15:04:41.677142    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c22df1abf8d3cdfe7d2732dc75b039fc8c515840a741a4f8293fe8243e65489\": container with ID starting with 2c22df1abf8d3cdfe7d2732dc75b039fc8c515840a741a4f8293fe8243e65489 not found: ID does not exist" containerID="2c22df1abf8d3cdfe7d2732dc75b039fc8c515840a741a4f8293fe8243e65489"
Jul 07 15:04:41.677230 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.677174    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c22df1abf8d3cdfe7d2732dc75b039fc8c515840a741a4f8293fe8243e65489"} err="failed to get container status \"2c22df1abf8d3cdfe7d2732dc75b039fc8c515840a741a4f8293fe8243e65489\": rpc error: code = NotFound desc = could not find container \"2c22df1abf8d3cdfe7d2732dc75b039fc8c515840a741a4f8293fe8243e65489\": container with ID starting with 2c22df1abf8d3cdfe7d2732dc75b039fc8c515840a741a4f8293fe8243e65489 not found: ID does not exist"
Jul 07 15:04:41.690133 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.690108    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"]
Jul 07 15:04:41.695261 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.695235    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/auth-test-model-26fa43-kserve-56c4974d8c-xmz4v"]
Jul 07 15:04:41.823002 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.822921    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:04:41.912721 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.912689    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-tmp-dir\") pod \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") "
Jul 07 15:04:41.912898 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.912730    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmjcx\" (UniqueName: \"kubernetes.io/projected/dbec51e9-1b4b-4101-8c7e-1a61b3918361-kube-api-access-qmjcx\") pod \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") "
Jul 07 15:04:41.912898 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.912796    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbec51e9-1b4b-4101-8c7e-1a61b3918361-tls-certs\") pod \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") "
Jul 07 15:04:41.912898 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.912830    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-model-cache\") pod \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") "
Jul 07 15:04:41.913069 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.912961    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-home\") pod \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") "
Jul 07 15:04:41.913069 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.913034    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dbec51e9-1b4b-4101-8c7e-1a61b3918361" (UID: "dbec51e9-1b4b-4101-8c7e-1a61b3918361"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:04:41.913069 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.913042    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-dshm\") pod \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\" (UID: \"dbec51e9-1b4b-4101-8c7e-1a61b3918361\") "
Jul 07 15:04:41.913245 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.913220    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-model-cache" (OuterVolumeSpecName: "model-cache") pod "dbec51e9-1b4b-4101-8c7e-1a61b3918361" (UID: "dbec51e9-1b4b-4101-8c7e-1a61b3918361"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:04:41.913311 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.913272    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-home" (OuterVolumeSpecName: "home") pod "dbec51e9-1b4b-4101-8c7e-1a61b3918361" (UID: "dbec51e9-1b4b-4101-8c7e-1a61b3918361"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:04:41.913366 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.913356    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-tmp-dir\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:04:41.913402 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.913375    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-model-cache\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:04:41.913402 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.913394    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-home\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:04:41.915144 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.915116    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-dshm" (OuterVolumeSpecName: "dshm") pod "dbec51e9-1b4b-4101-8c7e-1a61b3918361" (UID: "dbec51e9-1b4b-4101-8c7e-1a61b3918361"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:04:41.915144 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.915129    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbec51e9-1b4b-4101-8c7e-1a61b3918361-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dbec51e9-1b4b-4101-8c7e-1a61b3918361" (UID: "dbec51e9-1b4b-4101-8c7e-1a61b3918361"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:04:41.915297 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:41.915230    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbec51e9-1b4b-4101-8c7e-1a61b3918361-kube-api-access-qmjcx" (OuterVolumeSpecName: "kube-api-access-qmjcx") pod "dbec51e9-1b4b-4101-8c7e-1a61b3918361" (UID: "dbec51e9-1b4b-4101-8c7e-1a61b3918361"). InnerVolumeSpecName "kube-api-access-qmjcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:04:42.013858 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:42.013818    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbec51e9-1b4b-4101-8c7e-1a61b3918361-dshm\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:04:42.013858 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:42.013850    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmjcx\" (UniqueName: \"kubernetes.io/projected/dbec51e9-1b4b-4101-8c7e-1a61b3918361-kube-api-access-qmjcx\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:04:42.013858 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:42.013861    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbec51e9-1b4b-4101-8c7e-1a61b3918361-tls-certs\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:04:42.101056 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:42.100963    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2675923-2047-46b5-8337-65fd3a294e76" path="/var/lib/kubelet/pods/d2675923-2047-46b5-8337-65fd3a294e76/volumes"
Jul 07 15:04:42.673292 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:42.673256    2583 generic.go:356] "Generic (PLEG): container finished" podID="dbec51e9-1b4b-4101-8c7e-1a61b3918361" containerID="75d2ac6d5f40eed4b4b658bd5994ddab3a598cc82f06fc4530bea2ec7fc077d4" exitCode=0
Jul 07 15:04:42.673788 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:42.673310    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"
Jul 07 15:04:42.673788 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:42.673343    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279" event={"ID":"dbec51e9-1b4b-4101-8c7e-1a61b3918361","Type":"ContainerDied","Data":"75d2ac6d5f40eed4b4b658bd5994ddab3a598cc82f06fc4530bea2ec7fc077d4"}
Jul 07 15:04:42.673788 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:42.673382    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279" event={"ID":"dbec51e9-1b4b-4101-8c7e-1a61b3918361","Type":"ContainerDied","Data":"b79d7a389dc75d7c4bc0ed1d327f89a0313b5e2f967bac6d54b7660784726056"}
Jul 07 15:04:42.673788 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:42.673404    2583 scope.go:117] "RemoveContainer" containerID="75d2ac6d5f40eed4b4b658bd5994ddab3a598cc82f06fc4530bea2ec7fc077d4"
Jul 07 15:04:42.681543 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:42.681525    2583 scope.go:117] "RemoveContainer" containerID="75d2ac6d5f40eed4b4b658bd5994ddab3a598cc82f06fc4530bea2ec7fc077d4"
Jul 07 15:04:42.681801 ip-10-0-129-1 kubenswrapper[2583]: E0707 15:04:42.681779    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75d2ac6d5f40eed4b4b658bd5994ddab3a598cc82f06fc4530bea2ec7fc077d4\": container with ID starting with 75d2ac6d5f40eed4b4b658bd5994ddab3a598cc82f06fc4530bea2ec7fc077d4 not found: ID does not exist" containerID="75d2ac6d5f40eed4b4b658bd5994ddab3a598cc82f06fc4530bea2ec7fc077d4"
Jul 07 15:04:42.681859 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:42.681813    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75d2ac6d5f40eed4b4b658bd5994ddab3a598cc82f06fc4530bea2ec7fc077d4"} err="failed to get container status \"75d2ac6d5f40eed4b4b658bd5994ddab3a598cc82f06fc4530bea2ec7fc077d4\": rpc error: code = NotFound desc = could not find container \"75d2ac6d5f40eed4b4b658bd5994ddab3a598cc82f06fc4530bea2ec7fc077d4\": container with ID starting with 75d2ac6d5f40eed4b4b658bd5994ddab3a598cc82f06fc4530bea2ec7fc077d4 not found: ID does not exist"
Jul 07 15:04:42.687856 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:42.687829    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"]
Jul 07 15:04:42.693649 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:42.693624    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/auth-test-model-2f8e54-kserve-67c86dc4f9-mf279"]
Jul 07 15:04:44.100069 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:44.100036    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbec51e9-1b4b-4101-8c7e-1a61b3918361" path="/var/lib/kubelet/pods/dbec51e9-1b4b-4101-8c7e-1a61b3918361/volumes"
Jul 07 15:04:47.798810 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:47.798749    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:47.799319 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:47.798828    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:47.811315 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:47.811287    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:48.710749 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:48.710722    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:04:48.844780 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:04:48.844744    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:05:00.730392 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.730356    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"]
Jul 07 15:05:00.730786 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.730657    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2675923-2047-46b5-8337-65fd3a294e76" containerName="main"
Jul 07 15:05:00.730786 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.730668    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2675923-2047-46b5-8337-65fd3a294e76" containerName="main"
Jul 07 15:05:00.730786 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.730685    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbec51e9-1b4b-4101-8c7e-1a61b3918361" containerName="main"
Jul 07 15:05:00.730786 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.730691    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbec51e9-1b4b-4101-8c7e-1a61b3918361" containerName="main"
Jul 07 15:05:00.730786 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.730744    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2675923-2047-46b5-8337-65fd3a294e76" containerName="main"
Jul 07 15:05:00.730786 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.730753    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbec51e9-1b4b-4101-8c7e-1a61b3918361" containerName="main"
Jul 07 15:05:00.735060 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.735041    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.737095 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.737061    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2f8e54\"/\"kube-root-ca.crt\""
Jul 07 15:05:00.737544 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.737522    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2f8e54\"/\"default-dockercfg-9vbt6\""
Jul 07 15:05:00.737637 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.737586    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2f8e54\"/\"openshift-service-ca.crt\""
Jul 07 15:05:00.737637 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.737593    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2f8e54\"/\"sub-test-model-2f8e54-kserve-self-signed-certs\""
Jul 07 15:05:00.747885 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.747852    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"]
Jul 07 15:05:00.869185 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.869143    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-dshm\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.869185 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.869200    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe0b573e-7cb6-42d7-99c4-ca53017da362-tls-certs\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.869462 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.869274    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ftp9\" (UniqueName: \"kubernetes.io/projected/fe0b573e-7cb6-42d7-99c4-ca53017da362-kube-api-access-6ftp9\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.869462 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.869345    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-home\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.869462 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.869400    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-model-cache\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.869462 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.869421    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-tmp-dir\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.969814 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.969778    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-home\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.969814 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.969818    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-model-cache\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.970083 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.969834    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-tmp-dir\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.970083 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.969959    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-dshm\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.970083 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.970033    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe0b573e-7cb6-42d7-99c4-ca53017da362-tls-certs\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.970083 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.970067    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ftp9\" (UniqueName: \"kubernetes.io/projected/fe0b573e-7cb6-42d7-99c4-ca53017da362-kube-api-access-6ftp9\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.970303 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.970281    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-home\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.970368 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.970344    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-model-cache\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.970405 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.970353    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-tmp-dir\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.972247 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.972222    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-dshm\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.972617 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.972598    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe0b573e-7cb6-42d7-99c4-ca53017da362-tls-certs\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:00.978622 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:00.978604    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ftp9\" (UniqueName: \"kubernetes.io/projected/fe0b573e-7cb6-42d7-99c4-ca53017da362-kube-api-access-6ftp9\") pod \"sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") " pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:01.044748 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:01.044654    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:01.191636 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:01.191604    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"]
Jul 07 15:05:01.195167 ip-10-0-129-1 kubenswrapper[2583]: W0707 15:05:01.195142    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe0b573e_7cb6_42d7_99c4_ca53017da362.slice/crio-6cd1aa5c68d935f7c0f04a115139ea47429eff06dcba5f29dea67278c855abf3 WatchSource:0}: Error finding container 6cd1aa5c68d935f7c0f04a115139ea47429eff06dcba5f29dea67278c855abf3: Status 404 returned error can't find the container with id 6cd1aa5c68d935f7c0f04a115139ea47429eff06dcba5f29dea67278c855abf3
Jul 07 15:05:01.760174 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:01.760138    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl" event={"ID":"fe0b573e-7cb6-42d7-99c4-ca53017da362","Type":"ContainerStarted","Data":"8813e64d26ffeeef1c006042fba4add53b7281a63e3965104ae65e0d86a39f80"}
Jul 07 15:05:01.760174 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:01.760181    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl" event={"ID":"fe0b573e-7cb6-42d7-99c4-ca53017da362","Type":"ContainerStarted","Data":"6cd1aa5c68d935f7c0f04a115139ea47429eff06dcba5f29dea67278c855abf3"}
Jul 07 15:05:01.782513 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:01.782461    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl" podStartSLOduration=1.782446784 podStartE2EDuration="1.782446784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:01.782034213 +0000 UTC m=+2316.244934155" watchObservedRunningTime="2026-07-07 15:05:01.782446784 +0000 UTC m=+2316.245346727"
Jul 07 15:05:05.104213 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:05.104178    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:05:11.045849 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:11.045806    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:11.046367 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:11.045871    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:11.058464 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:11.058427    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:11.815803 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:11.815766    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:19.957279 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:19.957193    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:05:37.580209 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:37.580178    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"]
Jul 07 15:05:38.552935 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:38.552897    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"]
Jul 07 15:05:43.042666 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:43.042622    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:05:52.599771 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.599705    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9" podUID="6b8bcc08-d711-4007-b532-a5516790b301" containerName="main" containerID="cri-o://e3de2dd1e6ceb699ed336cbfafd4063728c5f74fc53f82a74384a436694b706e" gracePeriod=45
Jul 07 15:05:52.852427 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.852360    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:05:52.922668 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.922627    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-tmp-dir\") pod \"6b8bcc08-d711-4007-b532-a5516790b301\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") "
Jul 07 15:05:52.922851 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.922680    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8bcc08-d711-4007-b532-a5516790b301-tls-certs\") pod \"6b8bcc08-d711-4007-b532-a5516790b301\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") "
Jul 07 15:05:52.922851 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.922720    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnddz\" (UniqueName: \"kubernetes.io/projected/6b8bcc08-d711-4007-b532-a5516790b301-kube-api-access-mnddz\") pod \"6b8bcc08-d711-4007-b532-a5516790b301\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") "
Jul 07 15:05:52.922851 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.922773    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-model-cache\") pod \"6b8bcc08-d711-4007-b532-a5516790b301\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") "
Jul 07 15:05:52.922851 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.922840    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-home\") pod \"6b8bcc08-d711-4007-b532-a5516790b301\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") "
Jul 07 15:05:52.923072 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.922887    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6b8bcc08-d711-4007-b532-a5516790b301" (UID: "6b8bcc08-d711-4007-b532-a5516790b301"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:05:52.923141 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.923106    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-model-cache" (OuterVolumeSpecName: "model-cache") pod "6b8bcc08-d711-4007-b532-a5516790b301" (UID: "6b8bcc08-d711-4007-b532-a5516790b301"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:05:52.923141 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.923130    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-home" (OuterVolumeSpecName: "home") pod "6b8bcc08-d711-4007-b532-a5516790b301" (UID: "6b8bcc08-d711-4007-b532-a5516790b301"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:05:52.923222 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.923163    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-tmp-dir\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:05:52.924906 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.924882    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b8bcc08-d711-4007-b532-a5516790b301-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6b8bcc08-d711-4007-b532-a5516790b301" (UID: "6b8bcc08-d711-4007-b532-a5516790b301"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:05:52.924983 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.924925    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b8bcc08-d711-4007-b532-a5516790b301-kube-api-access-mnddz" (OuterVolumeSpecName: "kube-api-access-mnddz") pod "6b8bcc08-d711-4007-b532-a5516790b301" (UID: "6b8bcc08-d711-4007-b532-a5516790b301"). InnerVolumeSpecName "kube-api-access-mnddz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:05:52.975891 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.975854    2583 generic.go:356] "Generic (PLEG): container finished" podID="6b8bcc08-d711-4007-b532-a5516790b301" containerID="e3de2dd1e6ceb699ed336cbfafd4063728c5f74fc53f82a74384a436694b706e" exitCode=0
Jul 07 15:05:52.975891 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.975895    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"
Jul 07 15:05:52.976130 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.975941    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9" event={"ID":"6b8bcc08-d711-4007-b532-a5516790b301","Type":"ContainerDied","Data":"e3de2dd1e6ceb699ed336cbfafd4063728c5f74fc53f82a74384a436694b706e"}
Jul 07 15:05:52.976130 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.976003    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9" event={"ID":"6b8bcc08-d711-4007-b532-a5516790b301","Type":"ContainerDied","Data":"e93bafd05f6b4da9834a696ded48ff69b27e61a96ebb9f0b51e352d352c129a9"}
Jul 07 15:05:52.976130 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.976033    2583 scope.go:117] "RemoveContainer" containerID="e3de2dd1e6ceb699ed336cbfafd4063728c5f74fc53f82a74384a436694b706e"
Jul 07 15:05:52.985233 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.985211    2583 scope.go:117] "RemoveContainer" containerID="e3de2dd1e6ceb699ed336cbfafd4063728c5f74fc53f82a74384a436694b706e"
Jul 07 15:05:52.985519 ip-10-0-129-1 kubenswrapper[2583]: E0707 15:05:52.985502    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3de2dd1e6ceb699ed336cbfafd4063728c5f74fc53f82a74384a436694b706e\": container with ID starting with e3de2dd1e6ceb699ed336cbfafd4063728c5f74fc53f82a74384a436694b706e not found: ID does not exist" containerID="e3de2dd1e6ceb699ed336cbfafd4063728c5f74fc53f82a74384a436694b706e"
Jul 07 15:05:52.985593 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:52.985527    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3de2dd1e6ceb699ed336cbfafd4063728c5f74fc53f82a74384a436694b706e"} err="failed to get container status \"e3de2dd1e6ceb699ed336cbfafd4063728c5f74fc53f82a74384a436694b706e\": rpc error: code = NotFound desc = could not find container \"e3de2dd1e6ceb699ed336cbfafd4063728c5f74fc53f82a74384a436694b706e\": container with ID starting with e3de2dd1e6ceb699ed336cbfafd4063728c5f74fc53f82a74384a436694b706e not found: ID does not exist"
Jul 07 15:05:53.024299 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.024267    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-dshm\") pod \"6b8bcc08-d711-4007-b532-a5516790b301\" (UID: \"6b8bcc08-d711-4007-b532-a5516790b301\") "
Jul 07 15:05:53.024560 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.024543    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8bcc08-d711-4007-b532-a5516790b301-tls-certs\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:05:53.024610 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.024568    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnddz\" (UniqueName: \"kubernetes.io/projected/6b8bcc08-d711-4007-b532-a5516790b301-kube-api-access-mnddz\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:05:53.024610 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.024585    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-model-cache\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:05:53.024610 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.024598    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-home\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:05:53.026306 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.026285    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-dshm" (OuterVolumeSpecName: "dshm") pod "6b8bcc08-d711-4007-b532-a5516790b301" (UID: "6b8bcc08-d711-4007-b532-a5516790b301"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:05:53.125573 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.125496    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b8bcc08-d711-4007-b532-a5516790b301-dshm\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:05:53.299810 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.299775    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"]
Jul 07 15:05:53.315876 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.315850    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-26fa43/sub-test-model-26fa43-kserve-58594bcf67-gb8g9"]
Jul 07 15:05:53.574537 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.574465    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl" podUID="fe0b573e-7cb6-42d7-99c4-ca53017da362" containerName="main" containerID="cri-o://8813e64d26ffeeef1c006042fba4add53b7281a63e3965104ae65e0d86a39f80" gracePeriod=45
Jul 07 15:05:53.822377 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.822353    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:53.829842 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.829786    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-tmp-dir\") pod \"fe0b573e-7cb6-42d7-99c4-ca53017da362\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") "
Jul 07 15:05:53.829842 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.829819    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-home\") pod \"fe0b573e-7cb6-42d7-99c4-ca53017da362\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") "
Jul 07 15:05:53.830015 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.829840    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-model-cache\") pod \"fe0b573e-7cb6-42d7-99c4-ca53017da362\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") "
Jul 07 15:05:53.830015 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.829900    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-dshm\") pod \"fe0b573e-7cb6-42d7-99c4-ca53017da362\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") "
Jul 07 15:05:53.830015 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.829949    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ftp9\" (UniqueName: \"kubernetes.io/projected/fe0b573e-7cb6-42d7-99c4-ca53017da362-kube-api-access-6ftp9\") pod \"fe0b573e-7cb6-42d7-99c4-ca53017da362\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") "
Jul 07 15:05:53.830015 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.830013    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe0b573e-7cb6-42d7-99c4-ca53017da362-tls-certs\") pod \"fe0b573e-7cb6-42d7-99c4-ca53017da362\" (UID: \"fe0b573e-7cb6-42d7-99c4-ca53017da362\") "
Jul 07 15:05:53.830220 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.830098    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-home" (OuterVolumeSpecName: "home") pod "fe0b573e-7cb6-42d7-99c4-ca53017da362" (UID: "fe0b573e-7cb6-42d7-99c4-ca53017da362"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:05:53.830220 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.830120    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fe0b573e-7cb6-42d7-99c4-ca53017da362" (UID: "fe0b573e-7cb6-42d7-99c4-ca53017da362"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:05:53.830220 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.830161    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-model-cache" (OuterVolumeSpecName: "model-cache") pod "fe0b573e-7cb6-42d7-99c4-ca53017da362" (UID: "fe0b573e-7cb6-42d7-99c4-ca53017da362"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:05:53.830353 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.830259    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-tmp-dir\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:05:53.830353 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.830280    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-home\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:05:53.830353 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.830291    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-model-cache\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:05:53.832090 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.832069    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-dshm" (OuterVolumeSpecName: "dshm") pod "fe0b573e-7cb6-42d7-99c4-ca53017da362" (UID: "fe0b573e-7cb6-42d7-99c4-ca53017da362"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:05:53.832179 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.832101    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe0b573e-7cb6-42d7-99c4-ca53017da362-kube-api-access-6ftp9" (OuterVolumeSpecName: "kube-api-access-6ftp9") pod "fe0b573e-7cb6-42d7-99c4-ca53017da362" (UID: "fe0b573e-7cb6-42d7-99c4-ca53017da362"). InnerVolumeSpecName "kube-api-access-6ftp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:05:53.832179 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.832122    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe0b573e-7cb6-42d7-99c4-ca53017da362-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fe0b573e-7cb6-42d7-99c4-ca53017da362" (UID: "fe0b573e-7cb6-42d7-99c4-ca53017da362"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:05:53.931130 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.931094    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ftp9\" (UniqueName: \"kubernetes.io/projected/fe0b573e-7cb6-42d7-99c4-ca53017da362-kube-api-access-6ftp9\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:05:53.931130 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.931125    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe0b573e-7cb6-42d7-99c4-ca53017da362-tls-certs\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:05:53.931130 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.931133    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe0b573e-7cb6-42d7-99c4-ca53017da362-dshm\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:05:53.982374 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.982343    2583 generic.go:356] "Generic (PLEG): container finished" podID="fe0b573e-7cb6-42d7-99c4-ca53017da362" containerID="8813e64d26ffeeef1c006042fba4add53b7281a63e3965104ae65e0d86a39f80" exitCode=0
Jul 07 15:05:53.982527 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.982386    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"
Jul 07 15:05:53.982527 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.982398    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl" event={"ID":"fe0b573e-7cb6-42d7-99c4-ca53017da362","Type":"ContainerDied","Data":"8813e64d26ffeeef1c006042fba4add53b7281a63e3965104ae65e0d86a39f80"}
Jul 07 15:05:53.982527 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.982433    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl" event={"ID":"fe0b573e-7cb6-42d7-99c4-ca53017da362","Type":"ContainerDied","Data":"6cd1aa5c68d935f7c0f04a115139ea47429eff06dcba5f29dea67278c855abf3"}
Jul 07 15:05:53.982527 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.982451    2583 scope.go:117] "RemoveContainer" containerID="8813e64d26ffeeef1c006042fba4add53b7281a63e3965104ae65e0d86a39f80"
Jul 07 15:05:53.990892 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.990874    2583 scope.go:117] "RemoveContainer" containerID="8813e64d26ffeeef1c006042fba4add53b7281a63e3965104ae65e0d86a39f80"
Jul 07 15:05:53.991164 ip-10-0-129-1 kubenswrapper[2583]: E0707 15:05:53.991146    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8813e64d26ffeeef1c006042fba4add53b7281a63e3965104ae65e0d86a39f80\": container with ID starting with 8813e64d26ffeeef1c006042fba4add53b7281a63e3965104ae65e0d86a39f80 not found: ID does not exist" containerID="8813e64d26ffeeef1c006042fba4add53b7281a63e3965104ae65e0d86a39f80"
Jul 07 15:05:53.991217 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:53.991173    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8813e64d26ffeeef1c006042fba4add53b7281a63e3965104ae65e0d86a39f80"} err="failed to get container status \"8813e64d26ffeeef1c006042fba4add53b7281a63e3965104ae65e0d86a39f80\": rpc error: code = NotFound desc = could not find container \"8813e64d26ffeeef1c006042fba4add53b7281a63e3965104ae65e0d86a39f80\": container with ID starting with 8813e64d26ffeeef1c006042fba4add53b7281a63e3965104ae65e0d86a39f80 not found: ID does not exist"
Jul 07 15:05:54.004726 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:54.004703    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"]
Jul 07 15:05:54.009381 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:54.009362    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/sub-test-model-2f8e54-kserve-5dbddf8ff5-qfdnl"]
Jul 07 15:05:54.100153 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:54.100083    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b8bcc08-d711-4007-b532-a5516790b301" path="/var/lib/kubelet/pods/6b8bcc08-d711-4007-b532-a5516790b301/volumes"
Jul 07 15:05:54.100472 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:54.100459    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe0b573e-7cb6-42d7-99c4-ca53017da362" path="/var/lib/kubelet/pods/fe0b573e-7cb6-42d7-99c4-ca53017da362/volumes"
Jul 07 15:05:58.247451 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:05:58.247417    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:06:24.946502 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:24.946468    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"]
Jul 07 15:06:24.947036 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:24.946801    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b8bcc08-d711-4007-b532-a5516790b301" containerName="main"
Jul 07 15:06:24.947036 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:24.946812    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b8bcc08-d711-4007-b532-a5516790b301" containerName="main"
Jul 07 15:06:24.947036 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:24.946821    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe0b573e-7cb6-42d7-99c4-ca53017da362" containerName="main"
Jul 07 15:06:24.947036 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:24.946827    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe0b573e-7cb6-42d7-99c4-ca53017da362" containerName="main"
Jul 07 15:06:24.947036 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:24.946879    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b8bcc08-d711-4007-b532-a5516790b301" containerName="main"
Jul 07 15:06:24.947036 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:24.946887    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe0b573e-7cb6-42d7-99c4-ca53017da362" containerName="main"
Jul 07 15:06:24.949722 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:24.949703    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:24.951801 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:24.951779    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2f8e54\"/\"openshift-service-ca.crt\""
Jul 07 15:06:24.952535 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:24.952515    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2f8e54\"/\"default-dockercfg-9vbt6\""
Jul 07 15:06:24.952632 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:24.952525    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2f8e54\"/\"kube-root-ca.crt\""
Jul 07 15:06:24.952632 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:24.952577    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2f8e54\"/\"rate-test-model-2f8e54-kserve-self-signed-certs\""
Jul 07 15:06:24.962151 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:24.962126    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"]
Jul 07 15:06:25.094603 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.094565    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-home\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.094603 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.094606    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-model-cache\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.094870 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.094624    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-dshm\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.094870 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.094681    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-tls-certs\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.094870 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.094726    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-tmp-dir\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.094870 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.094805    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfmjt\" (UniqueName: \"kubernetes.io/projected/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-kube-api-access-xfmjt\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.195622 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.195583    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-home\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.195622 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.195627    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-model-cache\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.195892 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.195664    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-dshm\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.195892 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.195695    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-tls-certs\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.195892 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.195718    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-tmp-dir\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.195892 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.195797    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfmjt\" (UniqueName: \"kubernetes.io/projected/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-kube-api-access-xfmjt\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.196121 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.196063    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-home\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.196185 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.196163    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-model-cache\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.196245 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.196227    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-tmp-dir\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.198120 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.198060    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-dshm\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.198343 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.198326    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-tls-certs\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.203123 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.203097    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfmjt\" (UniqueName: \"kubernetes.io/projected/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-kube-api-access-xfmjt\") pod \"rate-test-model-2f8e54-kserve-67f49ff84c-4xglm\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") " pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.259381 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.259342    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:25.398916 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:25.398796    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"]
Jul 07 15:06:25.401788 ip-10-0-129-1 kubenswrapper[2583]: W0707 15:06:25.401751    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3b09b9a_3e3c_47ff_b551_54e1d9531cb7.slice/crio-e2249a315e49bc3e99e3626fd649f59647f8a23c8a90d69bb0031b35df4a1d54 WatchSource:0}: Error finding container e2249a315e49bc3e99e3626fd649f59647f8a23c8a90d69bb0031b35df4a1d54: Status 404 returned error can't find the container with id e2249a315e49bc3e99e3626fd649f59647f8a23c8a90d69bb0031b35df4a1d54
Jul 07 15:06:26.130710 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:26.130671    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm" event={"ID":"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7","Type":"ContainerStarted","Data":"2a50272b882b3474f15cd3b2e1da3c5cac1becfd52c882ff4104fe1f51c75621"}
Jul 07 15:06:26.130710 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:26.130715    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm" event={"ID":"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7","Type":"ContainerStarted","Data":"e2249a315e49bc3e99e3626fd649f59647f8a23c8a90d69bb0031b35df4a1d54"}
Jul 07 15:06:26.148604 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:26.148545    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm" podStartSLOduration=2.148525765 podStartE2EDuration="2.148525765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:06:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:06:26.147133426 +0000 UTC m=+2400.610033408" watchObservedRunningTime="2026-07-07 15:06:26.148525765 +0000 UTC m=+2400.611425738"
Jul 07 15:06:26.165932 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:26.165898    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 15:06:26.167620 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:26.167595    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h577t_c39143b3-6865-46e6-971c-9c33afb34b45/ovn-acl-logging/0.log"
Jul 07 15:06:28.944015 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:28.943961    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:06:35.260569 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:35.260521    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:35.261191 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:35.260588    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:35.273475 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:35.273446    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:36.183240 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:36.183202    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:06:48.946872 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:06:48.946782    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:07:02.738749 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:02.738714    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:07:10.947692 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:10.947656    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"]
Jul 07 15:07:15.938395 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:15.938352    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:07:25.969013 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:25.968958    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm" podUID="c3b09b9a-3e3c-47ff-b551-54e1d9531cb7" containerName="main" containerID="cri-o://2a50272b882b3474f15cd3b2e1da3c5cac1becfd52c882ff4104fe1f51c75621" gracePeriod=45
Jul 07 15:07:26.225743 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.225681    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:07:26.328915 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.328878    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-dshm\") pod \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") "
Jul 07 15:07:26.329149 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.328929    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-model-cache\") pod \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") "
Jul 07 15:07:26.329149 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.328949    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfmjt\" (UniqueName: \"kubernetes.io/projected/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-kube-api-access-xfmjt\") pod \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") "
Jul 07 15:07:26.329149 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.328972    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-home\") pod \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") "
Jul 07 15:07:26.329149 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.329056    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-tls-certs\") pod \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") "
Jul 07 15:07:26.329149 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.329092    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-tmp-dir\") pod \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\" (UID: \"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7\") "
Jul 07 15:07:26.329454 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.329291    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-model-cache" (OuterVolumeSpecName: "model-cache") pod "c3b09b9a-3e3c-47ff-b551-54e1d9531cb7" (UID: "c3b09b9a-3e3c-47ff-b551-54e1d9531cb7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:07:26.329454 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.329327    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-home" (OuterVolumeSpecName: "home") pod "c3b09b9a-3e3c-47ff-b551-54e1d9531cb7" (UID: "c3b09b9a-3e3c-47ff-b551-54e1d9531cb7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:07:26.329454 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.329398    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c3b09b9a-3e3c-47ff-b551-54e1d9531cb7" (UID: "c3b09b9a-3e3c-47ff-b551-54e1d9531cb7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:07:26.331203 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.331179    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-dshm" (OuterVolumeSpecName: "dshm") pod "c3b09b9a-3e3c-47ff-b551-54e1d9531cb7" (UID: "c3b09b9a-3e3c-47ff-b551-54e1d9531cb7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 15:07:26.331314 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.331289    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-kube-api-access-xfmjt" (OuterVolumeSpecName: "kube-api-access-xfmjt") pod "c3b09b9a-3e3c-47ff-b551-54e1d9531cb7" (UID: "c3b09b9a-3e3c-47ff-b551-54e1d9531cb7"). InnerVolumeSpecName "kube-api-access-xfmjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 15:07:26.331352 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.331289    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c3b09b9a-3e3c-47ff-b551-54e1d9531cb7" (UID: "c3b09b9a-3e3c-47ff-b551-54e1d9531cb7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 15:07:26.385050 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.385010    2583 generic.go:356] "Generic (PLEG): container finished" podID="c3b09b9a-3e3c-47ff-b551-54e1d9531cb7" containerID="2a50272b882b3474f15cd3b2e1da3c5cac1becfd52c882ff4104fe1f51c75621" exitCode=0
Jul 07 15:07:26.385050 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.385054    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"
Jul 07 15:07:26.385258 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.385064    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm" event={"ID":"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7","Type":"ContainerDied","Data":"2a50272b882b3474f15cd3b2e1da3c5cac1becfd52c882ff4104fe1f51c75621"}
Jul 07 15:07:26.385258 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.385102    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm" event={"ID":"c3b09b9a-3e3c-47ff-b551-54e1d9531cb7","Type":"ContainerDied","Data":"e2249a315e49bc3e99e3626fd649f59647f8a23c8a90d69bb0031b35df4a1d54"}
Jul 07 15:07:26.385258 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.385124    2583 scope.go:117] "RemoveContainer" containerID="2a50272b882b3474f15cd3b2e1da3c5cac1becfd52c882ff4104fe1f51c75621"
Jul 07 15:07:26.393709 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.393689    2583 scope.go:117] "RemoveContainer" containerID="2a50272b882b3474f15cd3b2e1da3c5cac1becfd52c882ff4104fe1f51c75621"
Jul 07 15:07:26.394069 ip-10-0-129-1 kubenswrapper[2583]: E0707 15:07:26.394043    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a50272b882b3474f15cd3b2e1da3c5cac1becfd52c882ff4104fe1f51c75621\": container with ID starting with 2a50272b882b3474f15cd3b2e1da3c5cac1becfd52c882ff4104fe1f51c75621 not found: ID does not exist" containerID="2a50272b882b3474f15cd3b2e1da3c5cac1becfd52c882ff4104fe1f51c75621"
Jul 07 15:07:26.394149 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.394077    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a50272b882b3474f15cd3b2e1da3c5cac1becfd52c882ff4104fe1f51c75621"} err="failed to get container status \"2a50272b882b3474f15cd3b2e1da3c5cac1becfd52c882ff4104fe1f51c75621\": rpc error: code = NotFound desc = could not find container \"2a50272b882b3474f15cd3b2e1da3c5cac1becfd52c882ff4104fe1f51c75621\": container with ID starting with 2a50272b882b3474f15cd3b2e1da3c5cac1becfd52c882ff4104fe1f51c75621 not found: ID does not exist"
Jul 07 15:07:26.404726 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.404695    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"]
Jul 07 15:07:26.408262 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.408238    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm"]
Jul 07 15:07:26.430104 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.430069    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-tmp-dir\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:07:26.430104 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.430103    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-dshm\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:07:26.430271 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.430112    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-model-cache\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:07:26.430271 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.430123    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfmjt\" (UniqueName: \"kubernetes.io/projected/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-kube-api-access-xfmjt\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:07:26.430271 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.430132    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-home\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:07:26.430271 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:26.430140    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7-tls-certs\") on node \"ip-10-0-129-1.ec2.internal\" DevicePath \"\""
Jul 07 15:07:28.100678 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:28.100646    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3b09b9a-3e3c-47ff-b551-54e1d9531cb7" path="/var/lib/kubelet/pods/c3b09b9a-3e3c-47ff-b551-54e1d9531cb7/volumes"
Jul 07 15:07:31.173464 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:31.173407    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-2f8e54/rate-test-model-2f8e54-kserve-67f49ff84c-4xglm" podUID="c3b09b9a-3e3c-47ff-b551-54e1d9531cb7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 07 15:07:50.542023 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:50.541962    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-tpftx"]
Jul 07 15:07:56.488773 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:07:56.488734    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d8694fd4c-jp4mj_ee076dc5-aed4-45c6-b138-7691c0b183ce/authorino/0.log"
Jul 07 15:08:02.321362 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:02.321330    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cm5cr_ae372a71-bd3b-4cd6-a899-ecc7e3740d8d/server/0.log"
Jul 07 15:08:02.445986 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:02.445939    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fq86n_6597e7e8-e649-4d74-8d04-15c8fad9b423/manager/0.log"
Jul 07 15:08:02.570647 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:02.570620    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7868f6cbb6-6spbq_9673ca4b-abb2-4005-897f-e7fa928de2ab/copy-manifests/0.log"
Jul 07 15:08:02.578639 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:02.578573    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7868f6cbb6-6spbq_9673ca4b-abb2-4005-897f-e7fa928de2ab/manager/0.log"
Jul 07 15:08:04.373442 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:04.373412    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d8694fd4c-jp4mj_ee076dc5-aed4-45c6-b138-7691c0b183ce/authorino/0.log"
Jul 07 15:08:04.720026 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:04.719970    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-qg4ln_edf0a916-c32a-47a4-a4dc-2a12e37b0d3e/kuadrant-console-plugin/0.log"
Jul 07 15:08:05.101031 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:05.100934    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-tpftx_012f69d1-b881-4733-af1b-d899441d50e9/limitador/0.log"
Jul 07 15:08:05.823594 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:05.823559    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7646845f6c-cdhzc_738fc825-7f18-4e35-b44e-b97bd47f9bf6/kube-auth-proxy/0.log"
Jul 07 15:08:07.356054 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:07.356024    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-xlq74_0a4806ad-14e7-4f78-b807-9c9853d1f18c/main/0.log"
Jul 07 15:08:11.054865 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.054829    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-btsff/must-gather-6t47s"]
Jul 07 15:08:11.055250 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.055180    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3b09b9a-3e3c-47ff-b551-54e1d9531cb7" containerName="main"
Jul 07 15:08:11.055250 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.055195    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3b09b9a-3e3c-47ff-b551-54e1d9531cb7" containerName="main"
Jul 07 15:08:11.055250 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.055247    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3b09b9a-3e3c-47ff-b551-54e1d9531cb7" containerName="main"
Jul 07 15:08:11.058393 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.058374    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-btsff/must-gather-6t47s"
Jul 07 15:08:11.060566 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.060541    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-btsff\"/\"default-dockercfg-kcwx6\""
Jul 07 15:08:11.061140 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.061121    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-btsff\"/\"kube-root-ca.crt\""
Jul 07 15:08:11.061239 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.061124    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-btsff\"/\"openshift-service-ca.crt\""
Jul 07 15:08:11.068366 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.068337    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-btsff/must-gather-6t47s"]
Jul 07 15:08:11.081471 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.081430    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmw9n\" (UniqueName: \"kubernetes.io/projected/c10bcb17-d7e2-4605-966c-48a25cf45614-kube-api-access-lmw9n\") pod \"must-gather-6t47s\" (UID: \"c10bcb17-d7e2-4605-966c-48a25cf45614\") " pod="openshift-must-gather-btsff/must-gather-6t47s"
Jul 07 15:08:11.081653 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.081594    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c10bcb17-d7e2-4605-966c-48a25cf45614-must-gather-output\") pod \"must-gather-6t47s\" (UID: \"c10bcb17-d7e2-4605-966c-48a25cf45614\") " pod="openshift-must-gather-btsff/must-gather-6t47s"
Jul 07 15:08:11.182388 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.182350    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c10bcb17-d7e2-4605-966c-48a25cf45614-must-gather-output\") pod \"must-gather-6t47s\" (UID: \"c10bcb17-d7e2-4605-966c-48a25cf45614\") " pod="openshift-must-gather-btsff/must-gather-6t47s"
Jul 07 15:08:11.182601 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.182433    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmw9n\" (UniqueName: \"kubernetes.io/projected/c10bcb17-d7e2-4605-966c-48a25cf45614-kube-api-access-lmw9n\") pod \"must-gather-6t47s\" (UID: \"c10bcb17-d7e2-4605-966c-48a25cf45614\") " pod="openshift-must-gather-btsff/must-gather-6t47s"
Jul 07 15:08:11.182780 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.182759    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c10bcb17-d7e2-4605-966c-48a25cf45614-must-gather-output\") pod \"must-gather-6t47s\" (UID: \"c10bcb17-d7e2-4605-966c-48a25cf45614\") " pod="openshift-must-gather-btsff/must-gather-6t47s"
Jul 07 15:08:11.193809 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.193784    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmw9n\" (UniqueName: \"kubernetes.io/projected/c10bcb17-d7e2-4605-966c-48a25cf45614-kube-api-access-lmw9n\") pod \"must-gather-6t47s\" (UID: \"c10bcb17-d7e2-4605-966c-48a25cf45614\") " pod="openshift-must-gather-btsff/must-gather-6t47s"
Jul 07 15:08:11.367964 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.367880    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-btsff/must-gather-6t47s"
Jul 07 15:08:11.500428 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.500400    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-btsff/must-gather-6t47s"]
Jul 07 15:08:11.502846 ip-10-0-129-1 kubenswrapper[2583]: W0707 15:08:11.502810    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc10bcb17_d7e2_4605_966c_48a25cf45614.slice/crio-8123e1bc88eba2fd7091ad1a53db585213cd00a90fbe5d714950d52c8a8dda72 WatchSource:0}: Error finding container 8123e1bc88eba2fd7091ad1a53db585213cd00a90fbe5d714950d52c8a8dda72: Status 404 returned error can't find the container with id 8123e1bc88eba2fd7091ad1a53db585213cd00a90fbe5d714950d52c8a8dda72
Jul 07 15:08:11.504751 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.504723    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 15:08:11.577832 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:11.577797    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-btsff/must-gather-6t47s" event={"ID":"c10bcb17-d7e2-4605-966c-48a25cf45614","Type":"ContainerStarted","Data":"8123e1bc88eba2fd7091ad1a53db585213cd00a90fbe5d714950d52c8a8dda72"}
Jul 07 15:08:13.600335 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:13.598128    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-btsff/must-gather-6t47s" event={"ID":"c10bcb17-d7e2-4605-966c-48a25cf45614","Type":"ContainerStarted","Data":"0ab184617aece0e4fc5b5240e6dbc2c3f80f19b88b7cbad4846636c11a9ee645"}
Jul 07 15:08:13.600335 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:13.598181    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-btsff/must-gather-6t47s" event={"ID":"c10bcb17-d7e2-4605-966c-48a25cf45614","Type":"ContainerStarted","Data":"a4badd991b58d4f564ffcc426e7523ea3980ac087defec2ef76da7b5c277acb9"}
Jul 07 15:08:13.615629 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:13.615567    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-btsff/must-gather-6t47s" podStartSLOduration=1.5852967580000001 podStartE2EDuration="2.615547652s" totalImagesPullingTime="1.030250894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:08:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:08:13.615071613 +0000 UTC m=+2508.077971568" watchObservedRunningTime="2026-07-07 15:08:13.615547652 +0000 UTC m=+2508.078447599"
Jul 07 15:08:14.437131 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:14.437101    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mk6nm_7e360a2f-655d-464f-98e3-00096c75fbe9/global-pull-secret-syncer/0.log"
Jul 07 15:08:14.608579 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:14.608532    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sr65q_06eab125-4856-449a-982f-111e9231f1d9/konnectivity-agent/0.log"
Jul 07 15:08:14.647333 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:14.647299    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-1.ec2.internal_603217e51a894b3722465edfd41817b7/haproxy/0.log"
Jul 07 15:08:19.845053 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:19.844812    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-d8694fd4c-jp4mj_ee076dc5-aed4-45c6-b138-7691c0b183ce/authorino/0.log"
Jul 07 15:08:19.983544 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:19.983512    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-qg4ln_edf0a916-c32a-47a4-a4dc-2a12e37b0d3e/kuadrant-console-plugin/0.log"
Jul 07 15:08:20.223030 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:20.222998    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-tpftx_012f69d1-b881-4733-af1b-d899441d50e9/limitador/0.log"
Jul 07 15:08:22.255198 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:22.255163    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5bvmm_2bb4fee1-b8d7-4c9b-be03-55193cc5d453/kube-state-metrics/0.log"
Jul 07 15:08:22.280235 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:22.280189    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5bvmm_2bb4fee1-b8d7-4c9b-be03-55193cc5d453/kube-rbac-proxy-main/0.log"
Jul 07 15:08:22.308409 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:22.308372    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5bvmm_2bb4fee1-b8d7-4c9b-be03-55193cc5d453/kube-rbac-proxy-self/0.log"
Jul 07 15:08:22.341218 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:22.341191    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-659d88c44-7n5lt_4b10b4b4-abd7-48a1-a9a0-821613845a27/metrics-server/0.log"
Jul 07 15:08:22.372323 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:22.372289    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-2jvm4_b1c71ac0-9506-4612-9bf0-d2f5ce17b1b9/monitoring-plugin/0.log"
Jul 07 15:08:22.501078 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:22.501047    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kthbj_b58ac55e-cbd5-4601-8be8-6aca24721652/node-exporter/0.log"
Jul 07 15:08:22.527417 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:22.527335    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kthbj_b58ac55e-cbd5-4601-8be8-6aca24721652/kube-rbac-proxy/0.log"
Jul 07 15:08:22.553323 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:22.553287    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kthbj_b58ac55e-cbd5-4601-8be8-6aca24721652/init-textfile/0.log"
Jul 07 15:08:22.966647 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:22.966619    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-5jdkl_456ac016-505d-4083-afce-53ae37023f6f/prometheus-operator/0.log"
Jul 07 15:08:23.008840 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:23.008807    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-5jdkl_456ac016-505d-4083-afce-53ae37023f6f/kube-rbac-proxy/0.log"
Jul 07 15:08:24.007696 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.007655    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"]
Jul 07 15:08:24.014674 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.014645    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.021933 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.021907    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"]
Jul 07 15:08:24.100295 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.100256    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/685f6a26-4889-4704-96ca-e8179ce30e64-lib-modules\") pod \"perf-node-gather-daemonset-779hp\" (UID: \"685f6a26-4889-4704-96ca-e8179ce30e64\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.100478 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.100419    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/685f6a26-4889-4704-96ca-e8179ce30e64-proc\") pod \"perf-node-gather-daemonset-779hp\" (UID: \"685f6a26-4889-4704-96ca-e8179ce30e64\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.100478 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.100459    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/685f6a26-4889-4704-96ca-e8179ce30e64-sys\") pod \"perf-node-gather-daemonset-779hp\" (UID: \"685f6a26-4889-4704-96ca-e8179ce30e64\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.100605 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.100531    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq44t\" (UniqueName: \"kubernetes.io/projected/685f6a26-4889-4704-96ca-e8179ce30e64-kube-api-access-tq44t\") pod \"perf-node-gather-daemonset-779hp\" (UID: \"685f6a26-4889-4704-96ca-e8179ce30e64\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.100605 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.100563    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/685f6a26-4889-4704-96ca-e8179ce30e64-podres\") pod \"perf-node-gather-daemonset-779hp\" (UID: \"685f6a26-4889-4704-96ca-e8179ce30e64\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.201150 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.201108    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/685f6a26-4889-4704-96ca-e8179ce30e64-proc\") pod \"perf-node-gather-daemonset-779hp\" (UID: \"685f6a26-4889-4704-96ca-e8179ce30e64\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.201351 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.201165    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/685f6a26-4889-4704-96ca-e8179ce30e64-sys\") pod \"perf-node-gather-daemonset-779hp\" (UID: \"685f6a26-4889-4704-96ca-e8179ce30e64\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.201351 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.201220    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq44t\" (UniqueName: \"kubernetes.io/projected/685f6a26-4889-4704-96ca-e8179ce30e64-kube-api-access-tq44t\") pod \"perf-node-gather-daemonset-779hp\" (UID: \"685f6a26-4889-4704-96ca-e8179ce30e64\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.201351 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.201238    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/685f6a26-4889-4704-96ca-e8179ce30e64-podres\") pod \"perf-node-gather-daemonset-779hp\" (UID: \"685f6a26-4889-4704-96ca-e8179ce30e64\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.201542 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.201347    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/685f6a26-4889-4704-96ca-e8179ce30e64-sys\") pod \"perf-node-gather-daemonset-779hp\" (UID: \"685f6a26-4889-4704-96ca-e8179ce30e64\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.201542 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.201355    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/685f6a26-4889-4704-96ca-e8179ce30e64-podres\") pod \"perf-node-gather-daemonset-779hp\" (UID: \"685f6a26-4889-4704-96ca-e8179ce30e64\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.201542 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.201418    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/685f6a26-4889-4704-96ca-e8179ce30e64-lib-modules\") pod \"perf-node-gather-daemonset-779hp\" (UID: \"685f6a26-4889-4704-96ca-e8179ce30e64\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.201725 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.201599    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/685f6a26-4889-4704-96ca-e8179ce30e64-proc\") pod \"perf-node-gather-daemonset-779hp\" (UID: \"685f6a26-4889-4704-96ca-e8179ce30e64\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.201725 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.201611    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/685f6a26-4889-4704-96ca-e8179ce30e64-lib-modules\") pod \"perf-node-gather-daemonset-779hp\" (UID: \"685f6a26-4889-4704-96ca-e8179ce30e64\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.217877 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.217840    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq44t\" (UniqueName: \"kubernetes.io/projected/685f6a26-4889-4704-96ca-e8179ce30e64-kube-api-access-tq44t\") pod \"perf-node-gather-daemonset-779hp\" (UID: \"685f6a26-4889-4704-96ca-e8179ce30e64\") " pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.332072 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.331985    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.507313 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.507279    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"]
Jul 07 15:08:24.510645 ip-10-0-129-1 kubenswrapper[2583]: W0707 15:08:24.510616    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod685f6a26_4889_4704_96ca_e8179ce30e64.slice/crio-85c76f99fe107a3314879ebb2c30f432846f99caeb3d94285c177c792af61079 WatchSource:0}: Error finding container 85c76f99fe107a3314879ebb2c30f432846f99caeb3d94285c177c792af61079: Status 404 returned error can't find the container with id 85c76f99fe107a3314879ebb2c30f432846f99caeb3d94285c177c792af61079
Jul 07 15:08:24.688549 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.688507    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp" event={"ID":"685f6a26-4889-4704-96ca-e8179ce30e64","Type":"ContainerStarted","Data":"f48648873451c6b1279d7899e3b54d94cf4c756f13f354bef4350a01015b467b"}
Jul 07 15:08:24.688751 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.688577    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp" event={"ID":"685f6a26-4889-4704-96ca-e8179ce30e64","Type":"ContainerStarted","Data":"85c76f99fe107a3314879ebb2c30f432846f99caeb3d94285c177c792af61079"}
Jul 07 15:08:24.689412 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.689382    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:24.708236 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:24.708163    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp" podStartSLOduration=1.7081401 podStartE2EDuration="1.7081401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:08:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:08:24.706834747 +0000 UTC m=+2519.169734686" watchObservedRunningTime="2026-07-07 15:08:24.7081401 +0000 UTC m=+2519.171040046"
Jul 07 15:08:27.443665 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:27.443633    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-95td9_fdf28b9f-131a-4f2a-94b0-a101c082c4b1/dns/0.log"
Jul 07 15:08:27.473907 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:27.473879    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-95td9_fdf28b9f-131a-4f2a-94b0-a101c082c4b1/kube-rbac-proxy/0.log"
Jul 07 15:08:27.598026 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:27.597994    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m2nzp_324585d5-c3f9-4d3a-97b6-01755df3f0bd/dns-node-resolver/0.log"
Jul 07 15:08:28.206671 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:28.206639    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-28k4c_13805be4-13ec-42d1-a93d-b7fff2258a24/node-ca/0.log"
Jul 07 15:08:29.549882 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:29.549854    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7646845f6c-cdhzc_738fc825-7f18-4e35-b44e-b97bd47f9bf6/kube-auth-proxy/0.log"
Jul 07 15:08:30.624914 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:30.624884    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9tpzt_160fe77b-4e72-4855-9eb0-ac0b2708177c/serve-healthcheck-canary/0.log"
Jul 07 15:08:31.288670 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:31.288643    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nqmhq_fc038d77-b57f-4354-a128-672f0d0b5319/kube-rbac-proxy/0.log"
Jul 07 15:08:31.314289 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:31.314254    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nqmhq_fc038d77-b57f-4354-a128-672f0d0b5319/exporter/0.log"
Jul 07 15:08:31.340546 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:31.340513    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nqmhq_fc038d77-b57f-4354-a128-672f0d0b5319/extractor/0.log"
Jul 07 15:08:31.710891 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:31.710848    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-btsff/perf-node-gather-daemonset-779hp"
Jul 07 15:08:34.338311 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:34.338231    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cm5cr_ae372a71-bd3b-4cd6-a899-ecc7e3740d8d/server/0.log"
Jul 07 15:08:34.419701 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:34.419670    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-fq86n_6597e7e8-e649-4d74-8d04-15c8fad9b423/manager/0.log"
Jul 07 15:08:34.450323 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:34.450290    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7868f6cbb6-6spbq_9673ca4b-abb2-4005-897f-e7fa928de2ab/manager/0.log"
Jul 07 15:08:34.475890 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:34.475868    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7868f6cbb6-6spbq_9673ca4b-abb2-4005-897f-e7fa928de2ab/copy-manifests/0.log"
Jul 07 15:08:34.950202 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:34.950175    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7646845f6c-cdhzc_738fc825-7f18-4e35-b44e-b97bd47f9bf6/kube-auth-proxy/0.log"
Jul 07 15:08:36.113356 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:36.113259    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-849574fc57-ff7t9_e2046f4a-8260-418a-9e3a-9f091aa95324/manager/0.log"
Jul 07 15:08:36.711890 ip-10-0-129-1 kubenswrapper[2583]: I0707 15:08:36.711866    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7646845f6c-cdhzc_738fc825-7f18-4e35-b44e-b97bd47f9bf6/kube-auth-proxy/0.log"