Jul 10 02:14:12.897781 ip-10-0-132-62 systemd[1]: Starting Kubernetes Kubelet...
Jul 10 02:14:13.318928 ip-10-0-132-62 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 02:14:13.318928 ip-10-0-132-62 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 10 02:14:13.318928 ip-10-0-132-62 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 02:14:13.318928 ip-10-0-132-62 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 10 02:14:13.318928 ip-10-0-132-62 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 02:14:13.319630 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.318979    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 10 02:14:13.324105 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324087    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:14:13.324105 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324104    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324108    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324111    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324114    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324117    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324120    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324123    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324126    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324129    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324131    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324134    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324137    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324139    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324142    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324145    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324147    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324150    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324152    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324155    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:14:13.324171 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324157    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324160    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324162    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324172    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324175    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324178    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324180    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324183    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324186    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324188    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324191    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324193    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324196    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324198    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324200    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324203    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324205    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324208    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324210    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324212    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324215    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:14:13.324676 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324217    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324220    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324222    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324226    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324230    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324233    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324236    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324239    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324241    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324244    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324247    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324252    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324255    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324258    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324261    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324264    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324266    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324268    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:14:13.325180 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324272    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324276    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324278    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324281    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324284    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324286    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324288    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324291    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324293    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324296    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324298    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324301    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324303    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324306    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324308    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324310    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324313    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324315    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324320    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324322    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:14:13.325647 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324325    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324327    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324330    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324332    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324335    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324337    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324340    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324344    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324347    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324349    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324352    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324354    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324357    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324359    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324362    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324364    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324367    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324369    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324372    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.324374    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:14:13.326121 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325113    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325120    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325123    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325127    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325130    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325134    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325137    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325139    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325142    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325144    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325147    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325150    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325153    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325155    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325158    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325160    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325163    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325166    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325168    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:14:13.326613 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325171    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325174    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325176    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325179    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325181    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325184    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325186    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325189    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325192    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325195    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325198    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325200    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325203    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325205    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325207    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325210    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325212    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325214    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325217    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325220    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:14:13.327109 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325224    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325227    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325230    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325232    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325235    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325238    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325240    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325243    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325248    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325251    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325254    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325258    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325261    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325263    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325266    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325268    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325271    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325274    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325276    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325279    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:14:13.327605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325281    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325284    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325286    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325289    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325291    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325293    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325296    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325299    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325302    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325304    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325307    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325309    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325311    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325314    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325316    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325319    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325321    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325323    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325326    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:14:13.328103 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325329    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325331    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325334    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325336    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325338    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325341    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325343    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325346    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325349    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325351    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325353    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325356    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325358    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325361    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325364    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325366    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325369    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325371    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325374    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325376    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:14:13.328605 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325379    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325854    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325860    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325863    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325866    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325869    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325871    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325876    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325879    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325882    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325885    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325888    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325891    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325895    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325899    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325902    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325905    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325910    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325912    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:14:13.329092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325915    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325917    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325920    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325923    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325925    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325928    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325931    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325933    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325936    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325938    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325941    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325943    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325946    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325949    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325952    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325954    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325957    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325959    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325962    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:14:13.329697 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325964    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325967    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325970    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325972    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325975    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325977    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325984    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325986    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325989    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325991    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325993    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.325998    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326000    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326003    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326005    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326008    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326010    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326013    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326016    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326018    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:14:13.330168 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326021    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326023    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326026    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326029    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326033    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326036    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326039    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326042    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326045    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326048    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326050    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326053    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326056    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326058    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326061    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326063    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326066    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326068    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326071    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:14:13.330686 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326074    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326077    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326079    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326081    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326085    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326088    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326091    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326093    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326096    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326098    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326100    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326103    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326105    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326108    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326110    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326112    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326115    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326118    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326123    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326126    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:14:13.331143 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326128    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326131    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326133    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326205    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326212    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326219    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326224    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326230    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326238    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326246    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326256    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326262    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326266    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326273    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326276    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326279    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326284    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326287    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326289    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326292    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326295    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326299    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326302    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326305    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 10 02:14:13.331636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326308    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326311    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326315    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326318    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326321    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326324    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326328    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326331    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326334    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326337    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326340    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326344    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326346    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326349    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326352    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326354    2579 flags.go:64] FLAG: --enable-server="true"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326357    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326364    2579 flags.go:64] FLAG: --event-burst="100"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326366    2579 flags.go:64] FLAG: --event-qps="50"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326369    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326372    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326375    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326385    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326388    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326393    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 10 02:14:13.332246 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326396    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326399    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326402    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326404    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326407    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326410    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326413    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326415    2579 flags.go:64] FLAG: --feature-gates=""
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326419    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326422    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326425    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326428    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326431    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326435    2579 flags.go:64] FLAG: --help="false"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326438    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326441    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326444    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326447    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326450    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326454    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326456    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326459    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326462    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 10 02:14:13.332881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326465    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326467    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326470    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326473    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326476    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326478    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326481    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326491    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326496    2579 flags.go:64] FLAG: --lock-file=""
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326512    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326516    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326519    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326524    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326527    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326529    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326532    2579 flags.go:64] FLAG: --logging-format="text"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326535    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326538    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326541    2579 flags.go:64] FLAG: --manifest-url=""
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326544    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326548    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326551    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326556    2579 flags.go:64] FLAG: --max-pods="110"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326559    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326562    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 10 02:14:13.333454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326565    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326568    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326571    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326574    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326576    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326583    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326587    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326590    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326593    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326596    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326601    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326604    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326607    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326610    2579 flags.go:64] FLAG: --port="10250"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326613    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326621    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dc965cda21375c88"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326627    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326630    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326633    2579 flags.go:64] FLAG: --register-node="true"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326635    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326639    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326642    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326645    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326647    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326651    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 10 02:14:13.334085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326654    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326657    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326660    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326663    2579 flags.go:64] FLAG: --runonce="false"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326667    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326670    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326673    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326676    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326678    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326681    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326684    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326687    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326690    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326692    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326695    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326698    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326700    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326703    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326706    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326712    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326714    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326717    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326721    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326732    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326735    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 10 02:14:13.334750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326738    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326741    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326743    2579 flags.go:64] FLAG: --v="2"
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326747    2579 flags.go:64] FLAG: --version="false"
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326751    2579 flags.go:64] FLAG: --vmodule=""
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326755    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.326758    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326878    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326882    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326885    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326888    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326892    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326896    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326899    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326902    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326905    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326907    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326910    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326912    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326915    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:14:13.335348 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326918    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326920    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326922    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326925    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326927    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326930    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326932    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326934    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326937    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326939    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326943    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326946    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326949    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326952    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326954    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326957    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326959    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326962    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326965    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326967    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326970    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:14:13.335881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326973    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326975    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326979    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326981    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326984    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326986    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326990    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326994    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.326997    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327000    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327002    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327005    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327008    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327011    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327013    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327016    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327019    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327022    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327024    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327028    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:14:13.336415 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327030    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327035    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327038    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327046    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327064    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327068    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327071    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327074    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327077    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327080    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327082    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327085    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327088    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327090    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327094    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327096    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327099    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327101    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327104    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:14:13.336926 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327106    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327109    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327112    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327114    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327117    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327119    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327123    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327127    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327129    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327132    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327135    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327138    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327141    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327143    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327147    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327149    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327152    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327156    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327159    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327162    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:14:13.337406 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327164    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:14:13.337913 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327167    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:14:13.337913 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327169    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:14:13.337913 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327172    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:14:13.337913 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327175    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:14:13.337913 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.327177    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:14:13.337913 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.327185    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 02:14:13.337913 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.334433    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 10 02:14:13.337913 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.334449    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 10 02:14:13.337913 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.335352    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 10 02:14:13.337913 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.335360    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 10 02:14:13.337913 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335430    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:14:13.337913 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335437    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:14:13.337913 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335440    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:14:13.337913 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335443    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:14:13.337913 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335446    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335449    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335451    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335454    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335459    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335463    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335466    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335469    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335472    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335474    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335477    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335480    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335482    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335485    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335494    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335496    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335499    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335519    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335522    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:14:13.338296 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335524    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335527    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335529    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335532    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335534    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335537    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335539    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335543    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335545    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335548    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335550    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335553    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335555    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335557    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335560    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335563    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335566    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335570    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335580    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335584    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:14:13.338771 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335586    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335589    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335591    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335594    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335597    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335599    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335602    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335604    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335612    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335615    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335618    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335621    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335624    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335626    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335629    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335631    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335634    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335636    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335639    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335641    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335644    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:14:13.339276 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335647    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335649    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335651    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335654    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335656    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335658    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335661    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335663    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335666    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335668    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335670    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335673    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335675    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335677    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335683    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335686    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335688    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335691    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335694    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:14:13.339803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335696    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:14:13.340259 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335704    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:14:13.340259 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335707    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:14:13.340259 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335710    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:14:13.340259 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335712    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:14:13.340259 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335715    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:14:13.340259 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335717    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:14:13.340259 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335720    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:14:13.340259 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335722    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:14:13.340259 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335725    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:14:13.340259 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335729    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:14:13.340259 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335732    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:14:13.340259 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335735    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:14:13.340259 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335738    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:14:13.340259 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335740    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:14:13.340259 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335742    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:14:13.340641 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.335747    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 02:14:13.340641 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335888    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:14:13.340641 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335893    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:14:13.340641 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335897    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:14:13.340641 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335900    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:14:13.340641 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335902    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:14:13.340641 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335906    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:14:13.340641 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335908    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:14:13.340641 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335911    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:14:13.340641 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335914    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:14:13.340641 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335917    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:14:13.340641 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335919    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:14:13.340641 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335922    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:14:13.340641 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335925    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335928    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335930    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335933    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335937    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335945    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335948    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335951    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335954    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335957    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335959    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335962    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335973    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335976    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335979    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335983    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335987    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335990    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335992    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:14:13.340986 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335995    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.335998    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336001    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336003    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336006    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336009    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336011    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336013    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336016    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336018    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336021    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336024    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336027    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336029    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336032    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336034    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336037    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336040    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336042    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336051    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:14:13.341447 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336054    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336056    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336058    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336061    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336063    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336066    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336068    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336070    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336073    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336075    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336077    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336080    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336082    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336085    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336088    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336090    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336093    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336095    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336098    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336100    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:14:13.342019 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336102    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336105    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336107    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336111    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336114    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336117    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336120    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336122    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336124    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336127    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336130    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336132    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336140    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336143    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336145    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336148    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336150    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336153    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336155    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336157    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:14:13.342498 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336160    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:14:13.342985 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336162    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:14:13.342985 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336165    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:14:13.342985 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336167    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:14:13.342985 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336169    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:14:13.342985 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336172    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:14:13.342985 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336174    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:14:13.342985 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:13.336177    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:14:13.342985 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.336182    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 02:14:13.342985 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.336324    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 10 02:14:13.342985 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.338529    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 10 02:14:13.342985 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.340079    2579 server.go:1019] "Starting client certificate rotation"
Jul 10 02:14:13.342985 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.340186    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 02:14:13.342985 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.340221    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 02:14:13.343331 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.343161    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 02:14:13.347830 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.347807    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 02:14:13.366028 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.366003    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 10 02:14:13.369072 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.369049    2579 log.go:25] "Validated CRI v1 image API"
Jul 10 02:14:13.370060 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.370043    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 10 02:14:13.371604 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.371588    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 02:14:13.373297 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.373273    2579 fs.go:135] Filesystem UUIDs: map[5b827e11-6d44-4098-804c-84d3821b0c71:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b77b7747-76bd-40f8-9fbe-e0240767fdaf:/dev/nvme0n1p3]
Jul 10 02:14:13.373369 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.373297    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 10 02:14:13.378558 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.378428    2579 manager.go:216] Machine: {Timestamp:2026-07-10 02:14:13.377059768 +0000 UTC m=+0.359817445 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100165 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23c0033b2209aff6cd460812d82862 SystemUUID:ec23c003-3b22-09af-f6cd-460812d82862 BootID:3ad758dc-04e6-499b-8272-0d354d8fa78b Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:42:f3:b2:68:33 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:42:f3:b2:68:33 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:c3:80:aa:37:bb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 10 02:14:13.378558 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.378546    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 10 02:14:13.378718 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.378668    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 10 02:14:13.382085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.382052    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 10 02:14:13.382232 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.382088    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-62.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 10 02:14:13.382899 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.382889    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 10 02:14:13.382931 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.382903    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 10 02:14:13.382931 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.382915    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 02:14:13.383737 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.383725    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 02:14:13.384622 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.384611    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 10 02:14:13.387711 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.387696    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 02:14:13.387846 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.387837    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 10 02:14:13.387884 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.387877    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 10 02:14:13.387912 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.387894    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 10 02:14:13.387942 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.387913    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 10 02:14:13.387942 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.387924    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 10 02:14:13.387942 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.387935    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 10 02:14:13.388338 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.388327    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 02:14:13.388376 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.388343    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 02:14:13.390826 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.390806    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 10 02:14:13.392241 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.392227    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 10 02:14:13.392297 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.392257    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 10 02:14:13.392927 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.392916    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 10 02:14:13.392960 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.392933    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 10 02:14:13.392960 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.392938    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 10 02:14:13.392960 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.392945    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 10 02:14:13.392960 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.392952    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 10 02:14:13.392960 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.392957    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 10 02:14:13.393092 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.393020    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 10 02:14:13.393092 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.393025    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 10 02:14:13.393092 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.393031    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 10 02:14:13.393092 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.393036    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 10 02:14:13.393092 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.393044    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 10 02:14:13.393092 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.393059    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 10 02:14:13.393977 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.393966    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 10 02:14:13.393977 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.393975    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 10 02:14:13.397196 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.397180    2579 server.go:1268] "Started kubelet"
Jul 10 02:14:13.397315 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.397261    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 10 02:14:13.397370 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.397341    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 10 02:14:13.398210 ip-10-0-132-62 systemd[1]: Started Kubernetes Kubelet.
Jul 10 02:14:13.401187 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.398181    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 10 02:14:13.401490 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.401441    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 10 02:14:13.403880 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.403862    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 10 02:14:13.404811 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.404780    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 10 02:14:13.405470 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.405453    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-62.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 10 02:14:13.405600 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.405576    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 10 02:14:13.405669 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.405576    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-62.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 10 02:14:13.407755 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.407740    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 10 02:14:13.407835 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.407809    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 10 02:14:13.407883 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.407836    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-62.ec2.internal\" not found"
Jul 10 02:14:13.408371 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.408356    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 10 02:14:13.408371 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.408358    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 10 02:14:13.408531 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.408382    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 10 02:14:13.408531 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.408472    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 10 02:14:13.408531 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.408481    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 10 02:14:13.408832 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.408569    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dn4fg"
Jul 10 02:14:13.408832 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.408575    2579 factory.go:55] Registering systemd factory
Jul 10 02:14:13.408832 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.408617    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 10 02:14:13.408960 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.408839    2579 factory.go:153] Registering CRI-O factory
Jul 10 02:14:13.408960 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.408853    2579 factory.go:223] Registration of the crio container factory successfully
Jul 10 02:14:13.408960 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.408898    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 10 02:14:13.408960 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.408922    2579 factory.go:103] Registering Raw factory
Jul 10 02:14:13.408960 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.408936    2579 manager.go:1196] Started watching for new ooms in manager
Jul 10 02:14:13.409682 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.409666    2579 manager.go:349] Starting recovery of all containers
Jul 10 02:14:13.410440 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.408861    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-62.ec2.internal.18c0cbe8da7c68ac  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-62.ec2.internal,UID:ip-10-0-132-62.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-62.ec2.internal,},FirstTimestamp:2026-07-10 02:14:13.397153964 +0000 UTC m=+0.379911641,LastTimestamp:2026-07-10 02:14:13.397153964 +0000 UTC m=+0.379911641,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-62.ec2.internal,}"
Jul 10 02:14:13.415816 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.415793    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-62.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 10 02:14:13.415927 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.415817    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 10 02:14:13.417865 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.417841    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dn4fg"
Jul 10 02:14:13.419819 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.419742    2579 manager.go:354] Recovery completed
Jul 10 02:14:13.425287 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.425273    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:14:13.429203 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.429185    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:14:13.429273 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.429218    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:14:13.429273 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.429232    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:14:13.429785 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.429770    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 10 02:14:13.429785 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.429784    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 10 02:14:13.429903 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.429792    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:14:13.429903 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.429801    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 02:14:13.430992 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.430980    2579 policy_none.go:49] "None policy: Start"
Jul 10 02:14:13.431060 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.430995    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 10 02:14:13.431060 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.431009    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 10 02:14:13.431144 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.431099    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:14:13.431144 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.431121    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:14:13.431144 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.431132    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:14:13.432013 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.432001    2579 policy_none.go:47] "Start"
Jul 10 02:14:13.432064 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.431959    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-62.ec2.internal.18c0cbe8dc656ecc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-62.ec2.internal,UID:ip-10-0-132-62.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-62.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-62.ec2.internal,},FirstTimestamp:2026-07-10 02:14:13.429202636 +0000 UTC m=+0.411960314,LastTimestamp:2026-07-10 02:14:13.429202636 +0000 UTC m=+0.411960314,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-62.ec2.internal,}"
Jul 10 02:14:13.472161 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.472142    2579 manager.go:337] "Starting Device Plugin manager"
Jul 10 02:14:13.483401 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.472178    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 10 02:14:13.483401 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.472191    2579 server.go:87] "Starting device plugin registration server"
Jul 10 02:14:13.483401 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.472470    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 10 02:14:13.483401 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.472485    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 10 02:14:13.483401 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.472606    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 10 02:14:13.483401 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.472699    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 10 02:14:13.483401 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.472709    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 10 02:14:13.483401 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.473243    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 10 02:14:13.483401 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.473295    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-62.ec2.internal\" not found"
Jul 10 02:14:13.558866 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.558823    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 10 02:14:13.560229 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.560210    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 10 02:14:13.560304 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.560258    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 10 02:14:13.560304 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.560284    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 10 02:14:13.560383 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.560326    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 10 02:14:13.563050 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.563026    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:14:13.573261 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.573221    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:14:13.574200 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.574184    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:14:13.574260 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.574217    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:14:13.574260 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.574227    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:14:13.574260 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.574249    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.586391 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.586373    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.586454 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.586395    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-62.ec2.internal\": node \"ip-10-0-132-62.ec2.internal\" not found"
Jul 10 02:14:13.619769 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.619741    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-62.ec2.internal\" not found"
Jul 10 02:14:13.661049 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.661005    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-62.ec2.internal"]
Jul 10 02:14:13.661169 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.661099    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:14:13.662108 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.662084    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:14:13.662161 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.662119    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:14:13.662161 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.662132    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:14:13.663684 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.663665    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:14:13.663853 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.663839    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.663904 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.663871    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:14:13.665137 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.665118    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:14:13.665225 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.665147    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:14:13.665225 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.665157    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:14:13.665225 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.665118    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:14:13.665225 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.665224    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:14:13.665368 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.665235    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:14:13.666431 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.666415    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.666525 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.666445    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:14:13.667938 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.667923    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:14:13.668026 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.667950    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:14:13.668026 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.667961    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:14:13.694187 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.694158    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-62.ec2.internal\" not found" node="ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.698402 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.698383    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-62.ec2.internal\" not found" node="ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.720032 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.720007    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-62.ec2.internal\" not found"
Jul 10 02:14:13.809060 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.809026    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0a4d8fc7cfb66a98bfd5d9b14d0140aa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal\" (UID: \"0a4d8fc7cfb66a98bfd5d9b14d0140aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.809060 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.809056    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a4d8fc7cfb66a98bfd5d9b14d0140aa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal\" (UID: \"0a4d8fc7cfb66a98bfd5d9b14d0140aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.809241 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.809084    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3593c0baaa5f598fd63dfa27adb409e3-config\") pod \"kube-apiserver-proxy-ip-10-0-132-62.ec2.internal\" (UID: \"3593c0baaa5f598fd63dfa27adb409e3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.821103 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.821084    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-62.ec2.internal\" not found"
Jul 10 02:14:13.910169 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.910111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a4d8fc7cfb66a98bfd5d9b14d0140aa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal\" (UID: \"0a4d8fc7cfb66a98bfd5d9b14d0140aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.910169 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.910144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3593c0baaa5f598fd63dfa27adb409e3-config\") pod \"kube-apiserver-proxy-ip-10-0-132-62.ec2.internal\" (UID: \"3593c0baaa5f598fd63dfa27adb409e3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.910169 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.910164    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0a4d8fc7cfb66a98bfd5d9b14d0140aa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal\" (UID: \"0a4d8fc7cfb66a98bfd5d9b14d0140aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.910301 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.910220    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0a4d8fc7cfb66a98bfd5d9b14d0140aa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal\" (UID: \"0a4d8fc7cfb66a98bfd5d9b14d0140aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.910301 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.910227    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3593c0baaa5f598fd63dfa27adb409e3-config\") pod \"kube-apiserver-proxy-ip-10-0-132-62.ec2.internal\" (UID: \"3593c0baaa5f598fd63dfa27adb409e3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.910301 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.910220    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a4d8fc7cfb66a98bfd5d9b14d0140aa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal\" (UID: \"0a4d8fc7cfb66a98bfd5d9b14d0140aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal"
Jul 10 02:14:13.921310 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:13.921292    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-62.ec2.internal\" not found"
Jul 10 02:14:13.997551 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:13.997525    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal"
Jul 10 02:14:14.001794 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.001420    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-62.ec2.internal"
Jul 10 02:14:14.021561 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:14.021540    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-62.ec2.internal\" not found"
Jul 10 02:14:14.122061 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:14.122025    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-62.ec2.internal\" not found"
Jul 10 02:14:14.222651 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:14.222577    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-62.ec2.internal\" not found"
Jul 10 02:14:14.323203 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:14.323173    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-62.ec2.internal\" not found"
Jul 10 02:14:14.339334 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.339314    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 10 02:14:14.339472 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.339451    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 02:14:14.353914 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.353889    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:14:14.354051 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.353983    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:14:14.388731 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.388696    2579 apiserver.go:52] "Watching apiserver"
Jul 10 02:14:14.396732 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.396702    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:14:14.398344 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.398325    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 10 02:14:14.402194 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.401939    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-mvjht","openshift-dns/node-resolver-9gnrj","openshift-image-registry/node-ca-mnqnz","openshift-multus/multus-additional-cni-plugins-m42l5","openshift-multus/network-metrics-daemon-dcc8z","openshift-network-diagnostics/network-check-target-frlf4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl","openshift-multus/multus-8dgnt","openshift-network-operator/iptables-alerter-jnblb","openshift-ovn-kubernetes/ovnkube-node-t2bng","kube-system/global-pull-secret-syncer-w59q2","kube-system/konnectivity-agent-wgkcm"]
Jul 10 02:14:14.403753 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.403735    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.404743 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.404723    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9gnrj"
Jul 10 02:14:14.405796 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.405780    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mnqnz"
Jul 10 02:14:14.406915 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.406902    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.407942 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.407910    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 10 02:14:14.407942 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.407937    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal"
Jul 10 02:14:14.408111 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.408045    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:14.408164 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.408110    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 10 02:14:14.408164 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:14.408122    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:14.408164 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.408135    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gzzf9\""
Jul 10 02:14:14.408304 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.408189    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 10 02:14:14.409041 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.409027    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 10 02:14:14.409248 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.409236    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:14.409309 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:14.409290    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:14.409708 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.409695    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 10 02:14:14.410305 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.410294    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.410496 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.410477    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 10 02:14:14.410756 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.410736    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 10 02:14:14.410823 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.410788    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 10 02:14:14.410823 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.410795    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s5ghr\""
Jul 10 02:14:14.410823 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.410810    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hc488\""
Jul 10 02:14:14.410948 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.410852    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 10 02:14:14.411033 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.411017    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 10 02:14:14.411127 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.411109    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 10 02:14:14.411175 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.411151    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 10 02:14:14.411216 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.411180    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rj5fp\""
Jul 10 02:14:14.411350 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.411338    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 10 02:14:14.411576 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.411563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.412387 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412372    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/79684caa-c963-4b62-8a74-ed4e87d3f201-host\") pod \"node-ca-mnqnz\" (UID: \"79684caa-c963-4b62-8a74-ed4e87d3f201\") " pod="openshift-image-registry/node-ca-mnqnz"
Jul 10 02:14:14.412464 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412415    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs\") pod \"network-metrics-daemon-dcc8z\" (UID: \"fafe8b6d-e09b-4dac-9687-43f5c7265be7\") " pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:14.412536 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412462    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cdca22f8-36eb-4162-abd9-9fed3a472a58-hosts-file\") pod \"node-resolver-9gnrj\" (UID: \"cdca22f8-36eb-4162-abd9-9fed3a472a58\") " pod="openshift-dns/node-resolver-9gnrj"
Jul 10 02:14:14.412536 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412496    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdca22f8-36eb-4162-abd9-9fed3a472a58-tmp-dir\") pod \"node-resolver-9gnrj\" (UID: \"cdca22f8-36eb-4162-abd9-9fed3a472a58\") " pod="openshift-dns/node-resolver-9gnrj"
Jul 10 02:14:14.412641 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412537    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wjsj\" (UniqueName: \"kubernetes.io/projected/cdca22f8-36eb-4162-abd9-9fed3a472a58-kube-api-access-5wjsj\") pod \"node-resolver-9gnrj\" (UID: \"cdca22f8-36eb-4162-abd9-9fed3a472a58\") " pod="openshift-dns/node-resolver-9gnrj"
Jul 10 02:14:14.412641 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412577    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/79684caa-c963-4b62-8a74-ed4e87d3f201-serviceca\") pod \"node-ca-mnqnz\" (UID: \"79684caa-c963-4b62-8a74-ed4e87d3f201\") " pod="openshift-image-registry/node-ca-mnqnz"
Jul 10 02:14:14.412641 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412618    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d64fac57-3910-4a19-8308-f56df2a18b8f-cnibin\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.412641 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412639    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-sysconfig\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.412821 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412655    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-tuned\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.412821 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412669    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d64fac57-3910-4a19-8308-f56df2a18b8f-cni-binary-copy\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.412821 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412698    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 10 02:14:14.412821 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412702    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-var-lib-kubelet\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.412821 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412730    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d64fac57-3910-4a19-8308-f56df2a18b8f-os-release\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.412821 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412747    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d64fac57-3910-4a19-8308-f56df2a18b8f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.412821 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412777    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6f86\" (UniqueName: \"kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86\") pod \"network-check-target-frlf4\" (UID: \"49e0bcec-546d-488b-8fa5-55a8d7589aca\") " pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:14.412821 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412796    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-modprobe-d\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.412821 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412812    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-systemd\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412833    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q5r9\" (UniqueName: \"kubernetes.io/projected/79684caa-c963-4b62-8a74-ed4e87d3f201-kube-api-access-2q5r9\") pod \"node-ca-mnqnz\" (UID: \"79684caa-c963-4b62-8a74-ed4e87d3f201\") " pod="openshift-image-registry/node-ca-mnqnz"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412848    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5gp5x\""
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412856    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d64fac57-3910-4a19-8308-f56df2a18b8f-system-cni-dir\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412887    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d64fac57-3910-4a19-8308-f56df2a18b8f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412907    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-host\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412969    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bq8s\" (UniqueName: \"kubernetes.io/projected/d64fac57-3910-4a19-8308-f56df2a18b8f-kube-api-access-2bq8s\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.413006    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-sysctl-d\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.412976    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jnblb"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.413022    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-lib-modules\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.413037    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc5tv\" (UniqueName: \"kubernetes.io/projected/fafe8b6d-e09b-4dac-9687-43f5c7265be7-kube-api-access-wc5tv\") pod \"network-metrics-daemon-dcc8z\" (UID: \"fafe8b6d-e09b-4dac-9687-43f5c7265be7\") " pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.413057    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-sysctl-conf\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.413074    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-run\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.413088    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-tmp\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.413102    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d64fac57-3910-4a19-8308-f56df2a18b8f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.413141    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-kubernetes\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.413166    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-sys\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.413214 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.413217    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpcx7\" (UniqueName: \"kubernetes.io/projected/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-kube-api-access-kpcx7\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.414185 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.414170    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.414678 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.414663    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 10 02:14:14.415836 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.415822    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:14.415895 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:14.415870    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:14.417563 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.417305    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wgkcm"
Jul 10 02:14:14.417563 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.417400    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 10 02:14:14.417969 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.417948    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 10 02:14:14.418055 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.417998    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 10 02:14:14.418055 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.418040    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 10 02:14:14.418154 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.418095    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 10 02:14:14.418582 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.418371    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 10 02:14:14.418582 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.418395    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 10 02:14:14.418582 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.418426    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 10 02:14:14.418776 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.418591    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-thlhx\""
Jul 10 02:14:14.418776 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.418685    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 10 02:14:14.418874 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.418821    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 10 02:14:14.418874 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.418847    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 10 02:14:14.418966 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.418824    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6nrd7\""
Jul 10 02:14:14.418966 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.418913    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bwtwv\""
Jul 10 02:14:14.423011 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.422969    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 02:09:14 +0000 UTC" deadline="2028-01-02 02:17:32.438481115 +0000 UTC"
Jul 10 02:14:14.423011 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.423011    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12984h3m18.015474044s"
Jul 10 02:14:14.425343 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.425315    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 10 02:14:14.426280 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.425774    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 10 02:14:14.426280 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.425794    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ft76h\""
Jul 10 02:14:14.426654 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.426633    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 02:14:14.426753 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.426731    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-62.ec2.internal"
Jul 10 02:14:14.428137 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.427366    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal"]
Jul 10 02:14:14.429866 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.429840    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 02:14:14.442141 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.442121    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 02:14:14.442230 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.442184    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-62.ec2.internal"]
Jul 10 02:14:14.451076 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.451058    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hdlnb"
Jul 10 02:14:14.463590 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.463566    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hdlnb"
Jul 10 02:14:14.509800 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.509774    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 10 02:14:14.513813 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.513791    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-systemd-units\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.513892 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.513831    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-sys\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.513892 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.513860    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpcx7\" (UniqueName: \"kubernetes.io/projected/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-kube-api-access-kpcx7\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.513892 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.513885    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs\") pod \"network-metrics-daemon-dcc8z\" (UID: \"fafe8b6d-e09b-4dac-9687-43f5c7265be7\") " pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:14.514013 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.513980    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-sys\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.514013 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:14.513991    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:14.514089 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.513989    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-multus-cni-dir\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.514134 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:14.514088    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs podName:fafe8b6d-e09b-4dac-9687-43f5c7265be7 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:15.014065322 +0000 UTC m=+1.996823010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs") pod "network-metrics-daemon-dcc8z" (UID: "fafe8b6d-e09b-4dac-9687-43f5c7265be7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:14.514134 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514124    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f95dj\" (UniqueName: \"kubernetes.io/projected/1007a2ac-fc5c-41b3-8b51-d1006101a828-kube-api-access-f95dj\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.514237 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-run-openvswitch\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.514237 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514173    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-cni-netd\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.514237 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514194    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdca22f8-36eb-4162-abd9-9fed3a472a58-tmp-dir\") pod \"node-resolver-9gnrj\" (UID: \"cdca22f8-36eb-4162-abd9-9fed3a472a58\") " pod="openshift-dns/node-resolver-9gnrj"
Jul 10 02:14:14.514237 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wjsj\" (UniqueName: \"kubernetes.io/projected/cdca22f8-36eb-4162-abd9-9fed3a472a58-kube-api-access-5wjsj\") pod \"node-resolver-9gnrj\" (UID: \"cdca22f8-36eb-4162-abd9-9fed3a472a58\") " pod="openshift-dns/node-resolver-9gnrj"
Jul 10 02:14:14.514417 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514344    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/79684caa-c963-4b62-8a74-ed4e87d3f201-serviceca\") pod \"node-ca-mnqnz\" (UID: \"79684caa-c963-4b62-8a74-ed4e87d3f201\") " pod="openshift-image-registry/node-ca-mnqnz"
Jul 10 02:14:14.514417 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514374    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d64fac57-3910-4a19-8308-f56df2a18b8f-cnibin\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.514537 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514417    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-socket-dir\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.514537 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-device-dir\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.514537 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514454    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d64fac57-3910-4a19-8308-f56df2a18b8f-cnibin\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.514537 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514475    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdca22f8-36eb-4162-abd9-9fed3a472a58-tmp-dir\") pod \"node-resolver-9gnrj\" (UID: \"cdca22f8-36eb-4162-abd9-9fed3a472a58\") " pod="openshift-dns/node-resolver-9gnrj"
Jul 10 02:14:14.514537 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514484    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-var-lib-cni-bin\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.514780 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-var-lib-cni-multus\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.514780 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514568    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-etc-kubernetes\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.514780 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514594    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-cni-bin\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.514780 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514617    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-ovnkube-script-lib\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.514780 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514648    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d64fac57-3910-4a19-8308-f56df2a18b8f-os-release\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.514780 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514722    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d64fac57-3910-4a19-8308-f56df2a18b8f-os-release\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.514780 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514724    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqt7n\" (UniqueName: \"kubernetes.io/projected/05535c21-295b-48a7-991c-c9538d0fc960-kube-api-access-dqt7n\") pod \"iptables-alerter-jnblb\" (UID: \"05535c21-295b-48a7-991c-c9538d0fc960\") " pod="openshift-network-operator/iptables-alerter-jnblb"
Jul 10 02:14:14.514780 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514763    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q5r9\" (UniqueName: \"kubernetes.io/projected/79684caa-c963-4b62-8a74-ed4e87d3f201-kube-api-access-2q5r9\") pod \"node-ca-mnqnz\" (UID: \"79684caa-c963-4b62-8a74-ed4e87d3f201\") " pod="openshift-image-registry/node-ca-mnqnz"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d64fac57-3910-4a19-8308-f56df2a18b8f-system-cni-dir\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514828    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d64fac57-3910-4a19-8308-f56df2a18b8f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514855    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-multus-socket-dir-parent\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514882    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1007a2ac-fc5c-41b3-8b51-d1006101a828-multus-daemon-config\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514887    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d64fac57-3910-4a19-8308-f56df2a18b8f-system-cni-dir\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514906    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-run-multus-certs\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514916    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/79684caa-c963-4b62-8a74-ed4e87d3f201-serviceca\") pod \"node-ca-mnqnz\" (UID: \"79684caa-c963-4b62-8a74-ed4e87d3f201\") " pod="openshift-image-registry/node-ca-mnqnz"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-log-socket\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514957    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-run-ovn-kubernetes\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.514985    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-host\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515019    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bq8s\" (UniqueName: \"kubernetes.io/projected/d64fac57-3910-4a19-8308-f56df2a18b8f-kube-api-access-2bq8s\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515042    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-run-k8s-cni-cncf-io\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515066    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-ovnkube-config\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515075    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-host\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515087    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-sys-fs\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515111    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-multus-conf-dir\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.515162 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/95280b80-3f30-46c4-be9b-33ff86d5ef24-dbus\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515174    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-run\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-tmp\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515240    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-os-release\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515260    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d64fac57-3910-4a19-8308-f56df2a18b8f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515268    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-slash\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515274    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-run\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515297    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/05535c21-295b-48a7-991c-c9538d0fc960-host-slash\") pod \"iptables-alerter-jnblb\" (UID: \"05535c21-295b-48a7-991c-c9538d0fc960\") " pod="openshift-network-operator/iptables-alerter-jnblb"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515364    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67f488cd-d334-4a42-b235-29879ebe70e0-konnectivity-ca\") pod \"konnectivity-agent-wgkcm\" (UID: \"67f488cd-d334-4a42-b235-29879ebe70e0\") " pod="kube-system/konnectivity-agent-wgkcm"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515386    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d64fac57-3910-4a19-8308-f56df2a18b8f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515402    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-kubernetes\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515415    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/79684caa-c963-4b62-8a74-ed4e87d3f201-host\") pod \"node-ca-mnqnz\" (UID: \"79684caa-c963-4b62-8a74-ed4e87d3f201\") " pod="openshift-image-registry/node-ca-mnqnz"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515430    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67f488cd-d334-4a42-b235-29879ebe70e0-agent-certs\") pod \"konnectivity-agent-wgkcm\" (UID: \"67f488cd-d334-4a42-b235-29879ebe70e0\") " pod="kube-system/konnectivity-agent-wgkcm"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515445    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cdca22f8-36eb-4162-abd9-9fed3a472a58-hosts-file\") pod \"node-resolver-9gnrj\" (UID: \"cdca22f8-36eb-4162-abd9-9fed3a472a58\") " pod="openshift-dns/node-resolver-9gnrj"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515467    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/95280b80-3f30-46c4-be9b-33ff86d5ef24-kubelet-config\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515473    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-kubernetes\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515490    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/79684caa-c963-4b62-8a74-ed4e87d3f201-host\") pod \"node-ca-mnqnz\" (UID: \"79684caa-c963-4b62-8a74-ed4e87d3f201\") " pod="openshift-image-registry/node-ca-mnqnz"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515491    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-sysconfig\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.516095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515525    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515550    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cdca22f8-36eb-4162-abd9-9fed3a472a58-hosts-file\") pod \"node-resolver-9gnrj\" (UID: \"cdca22f8-36eb-4162-abd9-9fed3a472a58\") " pod="openshift-dns/node-resolver-9gnrj"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515560    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d64fac57-3910-4a19-8308-f56df2a18b8f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515556    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-sysconfig\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515601    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-tuned\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515632    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d64fac57-3910-4a19-8308-f56df2a18b8f-cni-binary-copy\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-ovn-node-metrics-cert\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515728    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wpnw\" (UniqueName: \"kubernetes.io/projected/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-kube-api-access-7wpnw\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515784    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-var-lib-kubelet\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515825    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d64fac57-3910-4a19-8308-f56df2a18b8f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515831    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-var-lib-kubelet\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515876    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6f86\" (UniqueName: \"kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86\") pod \"network-check-target-frlf4\" (UID: \"49e0bcec-546d-488b-8fa5-55a8d7589aca\") " pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.516077    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d64fac57-3910-4a19-8308-f56df2a18b8f-cni-binary-copy\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.516383    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d64fac57-3910-4a19-8308-f56df2a18b8f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.515903    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-system-cni-dir\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.516647    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1007a2ac-fc5c-41b3-8b51-d1006101a828-cni-binary-copy\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.516677    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-run-netns\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.517118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.516737    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-env-overrides\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.516760    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-etc-selinux\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.516784    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-modprobe-d\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.516862    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-systemd\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.516889    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-var-lib-kubelet\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.516866    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-modprobe-d\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.516914    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-systemd\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.516920    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-kubelet\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.516944    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-run-netns\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.516962    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-etc-openvswitch\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.516986    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6qgm\" (UniqueName: \"kubernetes.io/projected/cae039bf-3da6-4997-9fbc-364a667782d8-kube-api-access-t6qgm\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517011    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/05535c21-295b-48a7-991c-c9538d0fc960-iptables-alerter-script\") pod \"iptables-alerter-jnblb\" (UID: \"05535c21-295b-48a7-991c-c9538d0fc960\") " pod="openshift-network-operator/iptables-alerter-jnblb"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517042    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-hostroot\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517080    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-var-lib-openvswitch\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517105    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-run-ovn\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517155    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-sysctl-d\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.518095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517217    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-lib-modules\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.518898 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517247    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc5tv\" (UniqueName: \"kubernetes.io/projected/fafe8b6d-e09b-4dac-9687-43f5c7265be7-kube-api-access-wc5tv\") pod \"network-metrics-daemon-dcc8z\" (UID: \"fafe8b6d-e09b-4dac-9687-43f5c7265be7\") " pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:14.518898 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517289    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-node-log\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.518898 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517315    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.518898 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517341    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-registration-dir\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.518898 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517351    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-lib-modules\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.518898 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517365    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-sysctl-conf\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.518898 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517387    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-cnibin\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.518898 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517408    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-run-systemd\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.518898 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517430    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:14.518898 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517626    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-sysctl-d\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.518898 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.517705    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-sysctl-conf\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.518898 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.518805    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-etc-tuned\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.519230 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.518909    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-tmp\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.526706 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:14.526554    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:14:14.526809 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:14.526738    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:14:14.526809 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:14.526784    2579 projected.go:196] Error preparing data for projected volume kube-api-access-q6f86 for pod openshift-network-diagnostics/network-check-target-frlf4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:14.526903 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:14.526852    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86 podName:49e0bcec-546d-488b-8fa5-55a8d7589aca nodeName:}" failed. No retries permitted until 2026-07-10 02:14:15.026826855 +0000 UTC m=+2.009584538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-q6f86" (UniqueName: "kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86") pod "network-check-target-frlf4" (UID: "49e0bcec-546d-488b-8fa5-55a8d7589aca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:14.527000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.526980    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q5r9\" (UniqueName: \"kubernetes.io/projected/79684caa-c963-4b62-8a74-ed4e87d3f201-kube-api-access-2q5r9\") pod \"node-ca-mnqnz\" (UID: \"79684caa-c963-4b62-8a74-ed4e87d3f201\") " pod="openshift-image-registry/node-ca-mnqnz"
Jul 10 02:14:14.527227 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.527198    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wjsj\" (UniqueName: \"kubernetes.io/projected/cdca22f8-36eb-4162-abd9-9fed3a472a58-kube-api-access-5wjsj\") pod \"node-resolver-9gnrj\" (UID: \"cdca22f8-36eb-4162-abd9-9fed3a472a58\") " pod="openshift-dns/node-resolver-9gnrj"
Jul 10 02:14:14.527712 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.527697    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpcx7\" (UniqueName: \"kubernetes.io/projected/e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20-kube-api-access-kpcx7\") pod \"tuned-mvjht\" (UID: \"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20\") " pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.528263 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.528246    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc5tv\" (UniqueName: \"kubernetes.io/projected/fafe8b6d-e09b-4dac-9687-43f5c7265be7-kube-api-access-wc5tv\") pod \"network-metrics-daemon-dcc8z\" (UID: \"fafe8b6d-e09b-4dac-9687-43f5c7265be7\") " pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:14.529038 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.529018    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bq8s\" (UniqueName: \"kubernetes.io/projected/d64fac57-3910-4a19-8308-f56df2a18b8f-kube-api-access-2bq8s\") pod \"multus-additional-cni-plugins-m42l5\" (UID: \"d64fac57-3910-4a19-8308-f56df2a18b8f\") " pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.567920 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.567903    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:14:14.618054 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618019    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-system-cni-dir\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.618054 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618051    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1007a2ac-fc5c-41b3-8b51-d1006101a828-cni-binary-copy\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.618231 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618067    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-run-netns\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.618231 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618090    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-env-overrides\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.618231 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618140    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-system-cni-dir\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.618231 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618204    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-run-netns\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.618436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618245    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-etc-selinux\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.618436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618269    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-var-lib-kubelet\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.618436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618284    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-kubelet\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.618436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618305    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-run-netns\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.618436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618329    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-etc-openvswitch\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.618436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618344    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-var-lib-kubelet\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.618436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618353    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6qgm\" (UniqueName: \"kubernetes.io/projected/cae039bf-3da6-4997-9fbc-364a667782d8-kube-api-access-t6qgm\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.618436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618377    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/05535c21-295b-48a7-991c-c9538d0fc960-iptables-alerter-script\") pod \"iptables-alerter-jnblb\" (UID: \"05535c21-295b-48a7-991c-c9538d0fc960\") " pod="openshift-network-operator/iptables-alerter-jnblb"
Jul 10 02:14:14.618436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618397    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-etc-openvswitch\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.618436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618399    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-run-netns\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.618436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618410    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-etc-selinux\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.618436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618399    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-hostroot\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.618436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618394    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-kubelet\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.618436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618433    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-hostroot\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618535    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-var-lib-openvswitch\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618560    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-run-ovn\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618596    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-env-overrides\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618589    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-var-lib-openvswitch\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618604    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618634    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-run-ovn\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618648    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618685    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-node-log\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618702    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1007a2ac-fc5c-41b3-8b51-d1006101a828-cni-binary-copy\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618731    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618736    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-node-log\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618760    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-registration-dir\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618782    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-cnibin\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618786    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-registration-dir\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-run-systemd\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618818    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618820    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-cnibin\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618833    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-systemd-units\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618846    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-run-systemd\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618862    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-multus-cni-dir\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618874    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-systemd-units\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618902    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f95dj\" (UniqueName: \"kubernetes.io/projected/1007a2ac-fc5c-41b3-8b51-d1006101a828-kube-api-access-f95dj\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618914    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-multus-cni-dir\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618906    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/05535c21-295b-48a7-991c-c9538d0fc960-iptables-alerter-script\") pod \"iptables-alerter-jnblb\" (UID: \"05535c21-295b-48a7-991c-c9538d0fc960\") " pod="openshift-network-operator/iptables-alerter-jnblb"
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618940    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-run-openvswitch\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:14.618942    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.618981    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-run-openvswitch\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:14.619000    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret podName:95280b80-3f30-46c4-be9b-33ff86d5ef24 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:15.118985547 +0000 UTC m=+2.101743210 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret") pod "global-pull-secret-syncer-w59q2" (UID: "95280b80-3f30-46c4-be9b-33ff86d5ef24") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619018    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-cni-netd\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619065    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-cni-netd\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619090    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-socket-dir\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619115    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-device-dir\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.619777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619140    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-var-lib-cni-bin\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619157    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-var-lib-cni-multus\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619171    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-etc-kubernetes\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619186    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-device-dir\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619192    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-cni-bin\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619210    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-var-lib-cni-bin\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619223    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-etc-kubernetes\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619227    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-var-lib-cni-multus\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619236    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-cni-bin\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619211    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-socket-dir\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619244    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-ovnkube-script-lib\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619274    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqt7n\" (UniqueName: \"kubernetes.io/projected/05535c21-295b-48a7-991c-c9538d0fc960-kube-api-access-dqt7n\") pod \"iptables-alerter-jnblb\" (UID: \"05535c21-295b-48a7-991c-c9538d0fc960\") " pod="openshift-network-operator/iptables-alerter-jnblb"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619303    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-multus-socket-dir-parent\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619327    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1007a2ac-fc5c-41b3-8b51-d1006101a828-multus-daemon-config\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619350    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-run-multus-certs\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619373    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-multus-socket-dir-parent\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619386    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-run-multus-certs\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619439    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-log-socket\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.620444 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619455    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-run-ovn-kubernetes\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619474    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-run-k8s-cni-cncf-io\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-log-socket\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619488    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-ovnkube-config\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619538    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-sys-fs\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619563    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-multus-conf-dir\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619587    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/95280b80-3f30-46c4-be9b-33ff86d5ef24-dbus\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619624    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-run-ovn-kubernetes\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619642    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-multus-conf-dir\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619646    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cae039bf-3da6-4997-9fbc-364a667782d8-sys-fs\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619685    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-host-run-k8s-cni-cncf-io\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619702    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/95280b80-3f30-46c4-be9b-33ff86d5ef24-dbus\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619718    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-os-release\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619745    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-slash\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619753    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-ovnkube-script-lib\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619768    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/05535c21-295b-48a7-991c-c9538d0fc960-host-slash\") pod \"iptables-alerter-jnblb\" (UID: \"05535c21-295b-48a7-991c-c9538d0fc960\") " pod="openshift-network-operator/iptables-alerter-jnblb"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619778    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1007a2ac-fc5c-41b3-8b51-d1006101a828-os-release\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619796    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67f488cd-d334-4a42-b235-29879ebe70e0-konnectivity-ca\") pod \"konnectivity-agent-wgkcm\" (UID: \"67f488cd-d334-4a42-b235-29879ebe70e0\") " pod="kube-system/konnectivity-agent-wgkcm"
Jul 10 02:14:14.621000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619806    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/05535c21-295b-48a7-991c-c9538d0fc960-host-slash\") pod \"iptables-alerter-jnblb\" (UID: \"05535c21-295b-48a7-991c-c9538d0fc960\") " pod="openshift-network-operator/iptables-alerter-jnblb"
Jul 10 02:14:14.621553 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619798    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-host-slash\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.621553 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619825    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1007a2ac-fc5c-41b3-8b51-d1006101a828-multus-daemon-config\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.621553 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619887    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67f488cd-d334-4a42-b235-29879ebe70e0-agent-certs\") pod \"konnectivity-agent-wgkcm\" (UID: \"67f488cd-d334-4a42-b235-29879ebe70e0\") " pod="kube-system/konnectivity-agent-wgkcm"
Jul 10 02:14:14.621553 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619916    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/95280b80-3f30-46c4-be9b-33ff86d5ef24-kubelet-config\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:14.621553 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619943    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-ovn-node-metrics-cert\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.621553 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619948    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/95280b80-3f30-46c4-be9b-33ff86d5ef24-kubelet-config\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:14.621553 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619915    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-ovnkube-config\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.621553 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.619978    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wpnw\" (UniqueName: \"kubernetes.io/projected/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-kube-api-access-7wpnw\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.621553 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.620223    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/67f488cd-d334-4a42-b235-29879ebe70e0-konnectivity-ca\") pod \"konnectivity-agent-wgkcm\" (UID: \"67f488cd-d334-4a42-b235-29879ebe70e0\") " pod="kube-system/konnectivity-agent-wgkcm"
Jul 10 02:14:14.622277 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.622251    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-ovn-node-metrics-cert\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.622390 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.622373    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/67f488cd-d334-4a42-b235-29879ebe70e0-agent-certs\") pod \"konnectivity-agent-wgkcm\" (UID: \"67f488cd-d334-4a42-b235-29879ebe70e0\") " pod="kube-system/konnectivity-agent-wgkcm"
Jul 10 02:14:14.627790 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.627756    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqt7n\" (UniqueName: \"kubernetes.io/projected/05535c21-295b-48a7-991c-c9538d0fc960-kube-api-access-dqt7n\") pod \"iptables-alerter-jnblb\" (UID: \"05535c21-295b-48a7-991c-c9538d0fc960\") " pod="openshift-network-operator/iptables-alerter-jnblb"
Jul 10 02:14:14.627987 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.627959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6qgm\" (UniqueName: \"kubernetes.io/projected/cae039bf-3da6-4997-9fbc-364a667782d8-kube-api-access-t6qgm\") pod \"aws-ebs-csi-driver-node-vcdnl\" (UID: \"cae039bf-3da6-4997-9fbc-364a667782d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.628157 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.628143    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f95dj\" (UniqueName: \"kubernetes.io/projected/1007a2ac-fc5c-41b3-8b51-d1006101a828-kube-api-access-f95dj\") pod \"multus-8dgnt\" (UID: \"1007a2ac-fc5c-41b3-8b51-d1006101a828\") " pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.628573 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.628559    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wpnw\" (UniqueName: \"kubernetes.io/projected/bdb1ec8c-059a-44c7-9f8b-03a01d4ae830-kube-api-access-7wpnw\") pod \"ovnkube-node-t2bng\" (UID: \"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830\") " pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.736973 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.736900    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mvjht"
Jul 10 02:14:14.745366 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:14.745320    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2b35c82_5ec5_4a6c_ac30_e1e3b60eda20.slice/crio-c558790df6543e19cb32a33660c0a7aedcc3c7eacb09fc3bc500a251511c484b WatchSource:0}: Error finding container c558790df6543e19cb32a33660c0a7aedcc3c7eacb09fc3bc500a251511c484b: Status 404 returned error can't find the container with id c558790df6543e19cb32a33660c0a7aedcc3c7eacb09fc3bc500a251511c484b
Jul 10 02:14:14.755264 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.755245    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9gnrj"
Jul 10 02:14:14.762683 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.762661    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mnqnz"
Jul 10 02:14:14.762928 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:14.762908    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdca22f8_36eb_4162_abd9_9fed3a472a58.slice/crio-e461b19adf7462d070429a728c7d048d97c38ee4fd6b14f5c6b32fd0eb1cde30 WatchSource:0}: Error finding container e461b19adf7462d070429a728c7d048d97c38ee4fd6b14f5c6b32fd0eb1cde30: Status 404 returned error can't find the container with id e461b19adf7462d070429a728c7d048d97c38ee4fd6b14f5c6b32fd0eb1cde30
Jul 10 02:14:14.769291 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:14.769266    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79684caa_c963_4b62_8a74_ed4e87d3f201.slice/crio-49d892d968731b234ceaed15221c53ef5fa37e10bd7f57d8d00fd7ae4af97336 WatchSource:0}: Error finding container 49d892d968731b234ceaed15221c53ef5fa37e10bd7f57d8d00fd7ae4af97336: Status 404 returned error can't find the container with id 49d892d968731b234ceaed15221c53ef5fa37e10bd7f57d8d00fd7ae4af97336
Jul 10 02:14:14.781598 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.781574    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-m42l5"
Jul 10 02:14:14.788110 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:14.788088    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd64fac57_3910_4a19_8308_f56df2a18b8f.slice/crio-a0cea6117e59b146c68af3ae4b8cc6a473c0a50cc0d058e8cde353b0670f5d61 WatchSource:0}: Error finding container a0cea6117e59b146c68af3ae4b8cc6a473c0a50cc0d058e8cde353b0670f5d61: Status 404 returned error can't find the container with id a0cea6117e59b146c68af3ae4b8cc6a473c0a50cc0d058e8cde353b0670f5d61
Jul 10 02:14:14.797950 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.797930    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl"
Jul 10 02:14:14.804807 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:14.804782    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae039bf_3da6_4997_9fbc_364a667782d8.slice/crio-5ebe9f9d4578f123cad7477764dc571341a6caa96ae1236025513b12f2adef20 WatchSource:0}: Error finding container 5ebe9f9d4578f123cad7477764dc571341a6caa96ae1236025513b12f2adef20: Status 404 returned error can't find the container with id 5ebe9f9d4578f123cad7477764dc571341a6caa96ae1236025513b12f2adef20
Jul 10 02:14:14.814117 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.814101    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8dgnt"
Jul 10 02:14:14.820736 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:14.820715    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1007a2ac_fc5c_41b3_8b51_d1006101a828.slice/crio-195ef52d4868d5445993c6d11b91852f5bea63e5c5398c183f5c99a3ebf97a98 WatchSource:0}: Error finding container 195ef52d4868d5445993c6d11b91852f5bea63e5c5398c183f5c99a3ebf97a98: Status 404 returned error can't find the container with id 195ef52d4868d5445993c6d11b91852f5bea63e5c5398c183f5c99a3ebf97a98
Jul 10 02:14:14.842770 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.842737    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jnblb"
Jul 10 02:14:14.849379 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:14.849360    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05535c21_295b_48a7_991c_c9538d0fc960.slice/crio-f747543b69ec16333ad2e8879ef65ed8c5f6b51d82af2c535902a7ac54c3b8be WatchSource:0}: Error finding container f747543b69ec16333ad2e8879ef65ed8c5f6b51d82af2c535902a7ac54c3b8be: Status 404 returned error can't find the container with id f747543b69ec16333ad2e8879ef65ed8c5f6b51d82af2c535902a7ac54c3b8be
Jul 10 02:14:14.868258 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.868237    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:14.871332 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:14.871317    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wgkcm"
Jul 10 02:14:14.877273 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:14.877254    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb1ec8c_059a_44c7_9f8b_03a01d4ae830.slice/crio-c68e51358665714a5b67542fc144501bb6dec5f54efb281137e73ce6bc923870 WatchSource:0}: Error finding container c68e51358665714a5b67542fc144501bb6dec5f54efb281137e73ce6bc923870: Status 404 returned error can't find the container with id c68e51358665714a5b67542fc144501bb6dec5f54efb281137e73ce6bc923870
Jul 10 02:14:14.879087 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:14.879065    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67f488cd_d334_4a42_b235_29879ebe70e0.slice/crio-8a435569358c60fb0f7b62529b250fa40ef62ae35a1eb2c996f9669018c2c968 WatchSource:0}: Error finding container 8a435569358c60fb0f7b62529b250fa40ef62ae35a1eb2c996f9669018c2c968: Status 404 returned error can't find the container with id 8a435569358c60fb0f7b62529b250fa40ef62ae35a1eb2c996f9669018c2c968
Jul 10 02:14:15.022538 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.022439    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs\") pod \"network-metrics-daemon-dcc8z\" (UID: \"fafe8b6d-e09b-4dac-9687-43f5c7265be7\") " pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:15.022673 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:15.022577    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:15.022673 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:15.022637    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs podName:fafe8b6d-e09b-4dac-9687-43f5c7265be7 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:16.022621095 +0000 UTC m=+3.005378760 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs") pod "network-metrics-daemon-dcc8z" (UID: "fafe8b6d-e09b-4dac-9687-43f5c7265be7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:15.123242 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.123202    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6f86\" (UniqueName: \"kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86\") pod \"network-check-target-frlf4\" (UID: \"49e0bcec-546d-488b-8fa5-55a8d7589aca\") " pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:15.123389 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.123270    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:15.123429 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:15.123389    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:15.123465 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:15.123448    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret podName:95280b80-3f30-46c4-be9b-33ff86d5ef24 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:16.123434162 +0000 UTC m=+3.106191827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret") pod "global-pull-secret-syncer-w59q2" (UID: "95280b80-3f30-46c4-be9b-33ff86d5ef24") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:15.123520 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:15.123486    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:14:15.123570 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:15.123530    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:14:15.123570 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:15.123544    2579 projected.go:196] Error preparing data for projected volume kube-api-access-q6f86 for pod openshift-network-diagnostics/network-check-target-frlf4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:15.123670 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:15.123600    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86 podName:49e0bcec-546d-488b-8fa5-55a8d7589aca nodeName:}" failed. No retries permitted until 2026-07-10 02:14:16.123582823 +0000 UTC m=+3.106340501 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-q6f86" (UniqueName: "kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86") pod "network-check-target-frlf4" (UID: "49e0bcec-546d-488b-8fa5-55a8d7589aca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:15.404830 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.404747    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:14:15.465414 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.465321    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 02:09:15 +0000 UTC" deadline="2027-12-06 14:50:36.336412485 +0000 UTC"
Jul 10 02:14:15.465414 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.465385    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12348h36m20.871032126s"
Jul 10 02:14:15.574220 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.574189    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:15.574365 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:15.574325    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:15.591410 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.591343    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jnblb" event={"ID":"05535c21-295b-48a7-991c-c9538d0fc960","Type":"ContainerStarted","Data":"f747543b69ec16333ad2e8879ef65ed8c5f6b51d82af2c535902a7ac54c3b8be"}
Jul 10 02:14:15.607041 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.606997    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8dgnt" event={"ID":"1007a2ac-fc5c-41b3-8b51-d1006101a828","Type":"ContainerStarted","Data":"195ef52d4868d5445993c6d11b91852f5bea63e5c5398c183f5c99a3ebf97a98"}
Jul 10 02:14:15.631456 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.631279    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m42l5" event={"ID":"d64fac57-3910-4a19-8308-f56df2a18b8f","Type":"ContainerStarted","Data":"a0cea6117e59b146c68af3ae4b8cc6a473c0a50cc0d058e8cde353b0670f5d61"}
Jul 10 02:14:15.640832 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.640792    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9gnrj" event={"ID":"cdca22f8-36eb-4162-abd9-9fed3a472a58","Type":"ContainerStarted","Data":"e461b19adf7462d070429a728c7d048d97c38ee4fd6b14f5c6b32fd0eb1cde30"}
Jul 10 02:14:15.649312 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.648903    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-62.ec2.internal" event={"ID":"3593c0baaa5f598fd63dfa27adb409e3","Type":"ContainerStarted","Data":"268dad4b1836cc773faf3d79ac0e37d61ee6da84d1be6ad6a7b12f367d0ca2e8"}
Jul 10 02:14:15.674794 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.674704    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal" event={"ID":"0a4d8fc7cfb66a98bfd5d9b14d0140aa","Type":"ContainerStarted","Data":"446c160a4448744e8ba5b29a39abd09e88bc512c8ae541ad26c900af84265b04"}
Jul 10 02:14:15.685306 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.685264    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wgkcm" event={"ID":"67f488cd-d334-4a42-b235-29879ebe70e0","Type":"ContainerStarted","Data":"8a435569358c60fb0f7b62529b250fa40ef62ae35a1eb2c996f9669018c2c968"}
Jul 10 02:14:15.708369 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.708324    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng" event={"ID":"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830","Type":"ContainerStarted","Data":"c68e51358665714a5b67542fc144501bb6dec5f54efb281137e73ce6bc923870"}
Jul 10 02:14:15.721223 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.721182    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl" event={"ID":"cae039bf-3da6-4997-9fbc-364a667782d8","Type":"ContainerStarted","Data":"5ebe9f9d4578f123cad7477764dc571341a6caa96ae1236025513b12f2adef20"}
Jul 10 02:14:15.746870 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.746828    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mnqnz" event={"ID":"79684caa-c963-4b62-8a74-ed4e87d3f201","Type":"ContainerStarted","Data":"49d892d968731b234ceaed15221c53ef5fa37e10bd7f57d8d00fd7ae4af97336"}
Jul 10 02:14:15.753461 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:15.753087    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mvjht" event={"ID":"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20","Type":"ContainerStarted","Data":"c558790df6543e19cb32a33660c0a7aedcc3c7eacb09fc3bc500a251511c484b"}
Jul 10 02:14:16.033064 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:16.032972    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs\") pod \"network-metrics-daemon-dcc8z\" (UID: \"fafe8b6d-e09b-4dac-9687-43f5c7265be7\") " pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:16.033248 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:16.033194    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:16.033315 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:16.033266    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs podName:fafe8b6d-e09b-4dac-9687-43f5c7265be7 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:18.033244852 +0000 UTC m=+5.016002523 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs") pod "network-metrics-daemon-dcc8z" (UID: "fafe8b6d-e09b-4dac-9687-43f5c7265be7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:16.134086 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:16.134046    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6f86\" (UniqueName: \"kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86\") pod \"network-check-target-frlf4\" (UID: \"49e0bcec-546d-488b-8fa5-55a8d7589aca\") " pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:16.134285 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:16.134119    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:16.134285 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:16.134251    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:16.134285 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:16.134276    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:14:16.134446 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:16.134302    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:14:16.134446 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:16.134317    2579 projected.go:196] Error preparing data for projected volume kube-api-access-q6f86 for pod openshift-network-diagnostics/network-check-target-frlf4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:16.134446 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:16.134322    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret podName:95280b80-3f30-46c4-be9b-33ff86d5ef24 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:18.134303193 +0000 UTC m=+5.117060863 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret") pod "global-pull-secret-syncer-w59q2" (UID: "95280b80-3f30-46c4-be9b-33ff86d5ef24") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:16.134446 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:16.134374    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86 podName:49e0bcec-546d-488b-8fa5-55a8d7589aca nodeName:}" failed. No retries permitted until 2026-07-10 02:14:18.13435662 +0000 UTC m=+5.117114301 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-q6f86" (UniqueName: "kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86") pod "network-check-target-frlf4" (UID: "49e0bcec-546d-488b-8fa5-55a8d7589aca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:16.465974 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:16.465876    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 02:09:15 +0000 UTC" deadline="2028-03-23 10:57:58.481494356 +0000 UTC"
Jul 10 02:14:16.465974 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:16.465922    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14936h43m42.015577045s"
Jul 10 02:14:16.561226 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:16.561193    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:16.561401 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:16.561340    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:16.561808 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:16.561785    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:16.561934 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:16.561912    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:17.564465 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:17.561201    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:17.564465 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:17.561603    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:18.054276 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:18.054001    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs\") pod \"network-metrics-daemon-dcc8z\" (UID: \"fafe8b6d-e09b-4dac-9687-43f5c7265be7\") " pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:18.054276 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:18.054170    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:18.054276 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:18.054230    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs podName:fafe8b6d-e09b-4dac-9687-43f5c7265be7 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:22.054211218 +0000 UTC m=+9.036968887 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs") pod "network-metrics-daemon-dcc8z" (UID: "fafe8b6d-e09b-4dac-9687-43f5c7265be7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:18.154693 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:18.154654    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6f86\" (UniqueName: \"kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86\") pod \"network-check-target-frlf4\" (UID: \"49e0bcec-546d-488b-8fa5-55a8d7589aca\") " pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:18.154881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:18.154730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:18.154881 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:18.154852    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:14:18.154999 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:18.154885    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:14:18.154999 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:18.154900    2579 projected.go:196] Error preparing data for projected volume kube-api-access-q6f86 for pod openshift-network-diagnostics/network-check-target-frlf4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:18.154999 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:18.154855    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:18.154999 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:18.154964    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86 podName:49e0bcec-546d-488b-8fa5-55a8d7589aca nodeName:}" failed. No retries permitted until 2026-07-10 02:14:22.154943547 +0000 UTC m=+9.137701215 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-q6f86" (UniqueName: "kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86") pod "network-check-target-frlf4" (UID: "49e0bcec-546d-488b-8fa5-55a8d7589aca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:18.155229 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:18.155016    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret podName:95280b80-3f30-46c4-be9b-33ff86d5ef24 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:22.154997808 +0000 UTC m=+9.137755473 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret") pod "global-pull-secret-syncer-w59q2" (UID: "95280b80-3f30-46c4-be9b-33ff86d5ef24") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:18.561589 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:18.561548    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:18.561793 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:18.561687    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:18.562149 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:18.562128    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:18.562276 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:18.562255    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:19.565658 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:19.565268    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:19.565658 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:19.565364    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:20.561748 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:20.560984    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:20.561748 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:20.561138    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:20.561748 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:20.561609    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:20.561748 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:20.561707    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:21.561146 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:21.561106    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:21.561638 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:21.561251    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:22.091300 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:22.091251    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs\") pod \"network-metrics-daemon-dcc8z\" (UID: \"fafe8b6d-e09b-4dac-9687-43f5c7265be7\") " pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:22.091482 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:22.091398    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:22.091482 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:22.091469    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs podName:fafe8b6d-e09b-4dac-9687-43f5c7265be7 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:30.091450126 +0000 UTC m=+17.074207810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs") pod "network-metrics-daemon-dcc8z" (UID: "fafe8b6d-e09b-4dac-9687-43f5c7265be7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:22.192679 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:22.192636    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6f86\" (UniqueName: \"kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86\") pod \"network-check-target-frlf4\" (UID: \"49e0bcec-546d-488b-8fa5-55a8d7589aca\") " pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:22.192851 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:22.192704    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:22.192899 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:22.192849    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:22.192932 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:22.192918    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret podName:95280b80-3f30-46c4-be9b-33ff86d5ef24 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:30.192896587 +0000 UTC m=+17.175654274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret") pod "global-pull-secret-syncer-w59q2" (UID: "95280b80-3f30-46c4-be9b-33ff86d5ef24") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:22.193355 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:22.193328    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:14:22.193413 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:22.193357    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:14:22.193413 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:22.193372    2579 projected.go:196] Error preparing data for projected volume kube-api-access-q6f86 for pod openshift-network-diagnostics/network-check-target-frlf4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:22.193541 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:22.193418    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86 podName:49e0bcec-546d-488b-8fa5-55a8d7589aca nodeName:}" failed. No retries permitted until 2026-07-10 02:14:30.19340318 +0000 UTC m=+17.176160864 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-q6f86" (UniqueName: "kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86") pod "network-check-target-frlf4" (UID: "49e0bcec-546d-488b-8fa5-55a8d7589aca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:22.561302 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:22.561175    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:22.561302 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:22.561180    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:22.561861 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:22.561307    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:22.561861 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:22.561403    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:23.562020 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:23.561599    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:23.562020 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:23.561728    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:24.560583 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:24.560541    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:24.560748 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:24.560677    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:24.561114 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:24.561086    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:24.561283 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:24.561209    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:25.560997 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:25.560957    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:25.561475 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:25.561107    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:26.560675 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:26.560629    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:26.560675 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:26.560671    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:26.560907 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:26.560776    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:26.560945 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:26.560913    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:27.562430 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:27.562404    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:27.562799 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:27.562561    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:27.795302 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:27.795233    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-62.ec2.internal" event={"ID":"3593c0baaa5f598fd63dfa27adb409e3","Type":"ContainerStarted","Data":"2f6ffcaae76b5cd7c7d7fa3d17534769978b8395057802bc7cf732123187c01d"}
Jul 10 02:14:27.811978 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:27.811924    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-62.ec2.internal" podStartSLOduration=12.811907098 podStartE2EDuration="12.811907098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:27.811270143 +0000 UTC m=+14.794027831" watchObservedRunningTime="2026-07-10 02:14:27.811907098 +0000 UTC m=+14.794664784"
Jul 10 02:14:28.560672 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:28.560638    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:28.560834 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:28.560768    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:28.560834 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:28.560805    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:28.560943 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:28.560887    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:28.799344 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:28.799133    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wgkcm" event={"ID":"67f488cd-d334-4a42-b235-29879ebe70e0","Type":"ContainerStarted","Data":"f3cd3c7fc9fd1357fddeb002b876419e84f599667cc97c2b3a0e7a3e3a372f21"}
Jul 10 02:14:28.800798 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:28.800773    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl" event={"ID":"cae039bf-3da6-4997-9fbc-364a667782d8","Type":"ContainerStarted","Data":"dc58f8d61be03a5c707229d0db582e7ab6e5276c5bab50660c8d362afe325b6e"}
Jul 10 02:14:28.802332 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:28.802311    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mnqnz" event={"ID":"79684caa-c963-4b62-8a74-ed4e87d3f201","Type":"ContainerStarted","Data":"a29637098bdebe9b5c0ad62804c9763b4778648913d205a0f930c71deec7a9d1"}
Jul 10 02:14:28.803996 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:28.803964    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m42l5" event={"ID":"d64fac57-3910-4a19-8308-f56df2a18b8f","Type":"ContainerStarted","Data":"500769a72e7252175a5721d704bde6fa924730250a0d771315eac222a5e904ff"}
Jul 10 02:14:28.805762 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:28.805734    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9gnrj" event={"ID":"cdca22f8-36eb-4162-abd9-9fed3a472a58","Type":"ContainerStarted","Data":"1943309c5990d93dd00b4f4a789342717d3809b6a2b72aca1285697078617319"}
Jul 10 02:14:28.807409 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:28.807386    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal" event={"ID":"0a4d8fc7cfb66a98bfd5d9b14d0140aa","Type":"ContainerStarted","Data":"91a951f0ac38dea20294669833c3794f1d11c9571e4b5200fd5042d0e7f1205c"}
Jul 10 02:14:28.836445 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:28.836371    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wgkcm" podStartSLOduration=2.209872274 podStartE2EDuration="14.836357559s" totalImagesPullingTime="12.626485285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:14:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:28.815926734 +0000 UTC m=+15.798684421" watchObservedRunningTime="2026-07-10 02:14:28.836357559 +0000 UTC m=+15.819115246"
Jul 10 02:14:28.836581 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:28.836557    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9gnrj" podStartSLOduration=2.071919179 podStartE2EDuration="14.836549758s" totalImagesPullingTime="12.764630579s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:28.836087395 +0000 UTC m=+15.818845080" watchObservedRunningTime="2026-07-10 02:14:28.836549758 +0000 UTC m=+15.819307445"
Jul 10 02:14:28.904828 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:28.904780    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mnqnz" podStartSLOduration=2.168445189 podStartE2EDuration="14.904761474s" totalImagesPullingTime="12.736316285s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:28.904695075 +0000 UTC m=+15.887452762" watchObservedRunningTime="2026-07-10 02:14:28.904761474 +0000 UTC m=+15.887519161"
Jul 10 02:14:29.561585 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:29.561549    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:29.561769 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:29.561670    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:30.161422 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:30.161381    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs\") pod \"network-metrics-daemon-dcc8z\" (UID: \"fafe8b6d-e09b-4dac-9687-43f5c7265be7\") " pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:30.161925 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:30.161575    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:30.161925 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:30.161654    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs podName:fafe8b6d-e09b-4dac-9687-43f5c7265be7 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:46.161630855 +0000 UTC m=+33.144388522 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs") pod "network-metrics-daemon-dcc8z" (UID: "fafe8b6d-e09b-4dac-9687-43f5c7265be7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:30.262323 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:30.262287    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6f86\" (UniqueName: \"kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86\") pod \"network-check-target-frlf4\" (UID: \"49e0bcec-546d-488b-8fa5-55a8d7589aca\") " pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:30.262553 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:30.262349    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:30.262553 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:30.262467    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:30.262553 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:30.262469    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:14:30.262553 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:30.262547    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:14:30.262742 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:30.262563    2579 projected.go:196] Error preparing data for projected volume kube-api-access-q6f86 for pod openshift-network-diagnostics/network-check-target-frlf4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:30.262742 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:30.262570    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret podName:95280b80-3f30-46c4-be9b-33ff86d5ef24 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:46.262554042 +0000 UTC m=+33.245311706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret") pod "global-pull-secret-syncer-w59q2" (UID: "95280b80-3f30-46c4-be9b-33ff86d5ef24") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:30.262742 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:30.262613    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86 podName:49e0bcec-546d-488b-8fa5-55a8d7589aca nodeName:}" failed. No retries permitted until 2026-07-10 02:14:46.26259703 +0000 UTC m=+33.245354717 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-q6f86" (UniqueName: "kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86") pod "network-check-target-frlf4" (UID: "49e0bcec-546d-488b-8fa5-55a8d7589aca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:30.560975 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:30.560905    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:30.561141 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:30.560911    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:30.561141 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:30.561046    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:30.561141 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:30.561129    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:31.561261 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:31.561222    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:31.561842 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:31.561347    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:31.813860 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:31.813787    2579 generic.go:356] "Generic (PLEG): container finished" podID="d64fac57-3910-4a19-8308-f56df2a18b8f" containerID="500769a72e7252175a5721d704bde6fa924730250a0d771315eac222a5e904ff" exitCode=0
Jul 10 02:14:31.814043 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:31.813858    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m42l5" event={"ID":"d64fac57-3910-4a19-8308-f56df2a18b8f","Type":"ContainerDied","Data":"500769a72e7252175a5721d704bde6fa924730250a0d771315eac222a5e904ff"}
Jul 10 02:14:31.815760 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:31.815735    2579 generic.go:356] "Generic (PLEG): container finished" podID="0a4d8fc7cfb66a98bfd5d9b14d0140aa" containerID="91a951f0ac38dea20294669833c3794f1d11c9571e4b5200fd5042d0e7f1205c" exitCode=0
Jul 10 02:14:31.815878 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:31.815776    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal" event={"ID":"0a4d8fc7cfb66a98bfd5d9b14d0140aa","Type":"ContainerDied","Data":"91a951f0ac38dea20294669833c3794f1d11c9571e4b5200fd5042d0e7f1205c"}
Jul 10 02:14:32.561210 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:32.561172    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:32.561394 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:32.561172    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:32.561394 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:32.561323    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:32.561924 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:32.561402    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:33.561545 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:33.561493    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:33.561961 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:33.561640    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:33.791682 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:33.791646    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wgkcm"
Jul 10 02:14:33.792290 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:33.792267    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wgkcm"
Jul 10 02:14:34.561521 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:34.561463    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:34.561690 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:34.561534    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:34.561690 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:34.561619    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:34.562048 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:34.561757    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:35.560857 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:35.560826    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:35.561036 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:35.560948    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:36.560755 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:36.560716    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:36.561344 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:36.560716    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:36.561344 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:36.560843    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:36.561344 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:36.560968    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:36.827871 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:36.827588    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8dgnt" event={"ID":"1007a2ac-fc5c-41b3-8b51-d1006101a828","Type":"ContainerStarted","Data":"2ebea9202454a3f86f6924f3628205281239edd0cea1da86207302d20396ea67"}
Jul 10 02:14:36.829622 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:36.829596    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal" event={"ID":"0a4d8fc7cfb66a98bfd5d9b14d0140aa","Type":"ContainerStarted","Data":"25781936135a34b49330a4d5cb9a954bacaea9176097f83742a66c42627e4fa7"}
Jul 10 02:14:36.832047 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:36.832021    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng" event={"ID":"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830","Type":"ContainerStarted","Data":"24fa63a269b70493a6c3a63dfe64b03b62fafdb5a23c1671ed4797480914ee98"}
Jul 10 02:14:36.833619 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:36.833598    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mvjht" event={"ID":"e2b35c82-5ec5-4a6c-ac30-e1e3b60eda20","Type":"ContainerStarted","Data":"c4d4fe080c7e578c5a97b89b6813684170abe923483a7069a653f0c7da83108d"}
Jul 10 02:14:36.847306 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:36.847265    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8dgnt" podStartSLOduration=0.997557371 podStartE2EDuration="22.847253383s" totalImagesPullingTime="21.849696012s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:36.847039131 +0000 UTC m=+23.829796816" watchObservedRunningTime="2026-07-10 02:14:36.847253383 +0000 UTC m=+23.830011069"
Jul 10 02:14:36.864194 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:36.864149    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mvjht" podStartSLOduration=10.065175268 podStartE2EDuration="22.86413614s" totalImagesPullingTime="12.798960872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:36.863925535 +0000 UTC m=+23.846683222" watchObservedRunningTime="2026-07-10 02:14:36.86413614 +0000 UTC m=+23.846893826"
Jul 10 02:14:36.880209 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:36.880165    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-62.ec2.internal" podStartSLOduration=21.880154082 podStartE2EDuration="21.880154082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:36.879792584 +0000 UTC m=+23.862550271" watchObservedRunningTime="2026-07-10 02:14:36.880154082 +0000 UTC m=+23.862911767"
Jul 10 02:14:37.037912 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:37.037888    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 10 02:14:37.487186 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:37.487039    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T02:14:37.037906619Z","UUID":"e86e246a-348a-4c2b-a05d-0823a28b21a1","Handler":null,"Name":"","Endpoint":""}
Jul 10 02:14:37.489172 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:37.489151    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 10 02:14:37.489306 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:37.489195    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 10 02:14:37.560755 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:37.560721    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:37.560938 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:37.560865    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:37.840342 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:37.840253    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng" event={"ID":"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830","Type":"ContainerStarted","Data":"bcf38b03547a53c3345be5830d46516126999666e174d9d3caa3830ed6ed33c0"}
Jul 10 02:14:37.840342 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:37.840336    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng" event={"ID":"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830","Type":"ContainerStarted","Data":"0488929d5f6501028abe19e157cd7a42610902c873678c5c00500c9bd4caeb1a"}
Jul 10 02:14:37.840659 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:37.840368    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng" event={"ID":"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830","Type":"ContainerStarted","Data":"5b48fd614bf9a0923a06c76061621e18c50f86b00a63c15dd10260fe6ba6d42d"}
Jul 10 02:14:37.840659 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:37.840407    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng" event={"ID":"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830","Type":"ContainerStarted","Data":"7e3bc4aa2e3f3af8acee9e2df6540c10d9697e6349f59c6564982a49ce60cd09"}
Jul 10 02:14:37.840659 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:37.840453    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng" event={"ID":"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830","Type":"ContainerStarted","Data":"d809e1231e3c63f95d5a8c0c82037bc8ea98beec396b022e45e9054f59899603"}
Jul 10 02:14:37.842681 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:37.842641    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl" event={"ID":"cae039bf-3da6-4997-9fbc-364a667782d8","Type":"ContainerStarted","Data":"8b0ab0dd4bcc36477530c728f01ebe386f53aa510d1b757cb4110aacca39f6a4"}
Jul 10 02:14:38.561248 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:38.561010    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:38.561741 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:38.561031    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:38.561741 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:38.561307    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:38.561741 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:38.561409    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:38.846308 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:38.846226    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jnblb" event={"ID":"05535c21-295b-48a7-991c-c9538d0fc960","Type":"ContainerStarted","Data":"972e75094d2d1be74d742ac84f8d7fcea2ac06751dfb808fbca054f5df4cf8f5"}
Jul 10 02:14:38.848085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:38.848061    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl" event={"ID":"cae039bf-3da6-4997-9fbc-364a667782d8","Type":"ContainerStarted","Data":"d1e676f9b25c004c7593eeaaead59a16945529a061d66e7a2855fe223e7876b1"}
Jul 10 02:14:38.877586 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:38.877540    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jnblb" podStartSLOduration=12.134277419 podStartE2EDuration="24.877527486s" totalImagesPullingTime="12.743250067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:38.877432988 +0000 UTC m=+25.860190674" watchObservedRunningTime="2026-07-10 02:14:38.877527486 +0000 UTC m=+25.860285171"
Jul 10 02:14:39.560519 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:39.560480    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:39.560648 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:39.560628    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:39.854032 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:39.853937    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng" event={"ID":"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830","Type":"ContainerStarted","Data":"aa4a9dc4898dea586704c02caeb1cd0d512265e53c22ec857a184e15cfe6c063"}
Jul 10 02:14:40.560765 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:40.560734    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:40.560939 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:40.560744    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:40.560939 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:40.560836    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:40.560939 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:40.560900    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:41.560703 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:41.560669    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:41.561275 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:41.560765    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:41.861377 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:41.861252    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng" event={"ID":"bdb1ec8c-059a-44c7-9f8b-03a01d4ae830","Type":"ContainerStarted","Data":"2864e3123d47546d36066572824396dadb157827cf8d2965ff70e01b7d070ef8"}
Jul 10 02:14:41.861483 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:41.861429    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:41.861632 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:41.861565    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:41.861734 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:41.861640    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:41.877858 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:41.877831    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:41.878031 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:41.878016    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:14:41.895328 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:41.895259    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vcdnl" podStartSLOduration=5.76389498 podStartE2EDuration="27.895243767s" totalImagesPullingTime="22.131348787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:38.910364116 +0000 UTC m=+25.893121802" watchObservedRunningTime="2026-07-10 02:14:41.895243767 +0000 UTC m=+28.878001476"
Jul 10 02:14:41.895516 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:41.895479    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng" podStartSLOduration=6.102358271 podStartE2EDuration="27.895472213s" totalImagesPullingTime="21.793113942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:41.895117242 +0000 UTC m=+28.877874928" watchObservedRunningTime="2026-07-10 02:14:41.895472213 +0000 UTC m=+28.878229898"
Jul 10 02:14:42.561142 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:42.561111    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:42.561816 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:42.561113    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:42.561816 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:42.561219    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:42.561816 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:42.561300    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:43.562265 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:43.562235    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:43.562653 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:43.562353    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:43.725281 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:43.725249    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dcc8z"]
Jul 10 02:14:43.725428 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:43.725384    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:43.725548 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:43.725520    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:43.729103 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:43.729066    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-frlf4"]
Jul 10 02:14:43.729212 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:43.729164    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:43.729275 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:43.729249    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:43.730431 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:43.729817    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w59q2"]
Jul 10 02:14:43.865834 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:43.865762    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:43.865989 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:43.865889    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:44.746552 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:44.746364    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wgkcm"
Jul 10 02:14:44.746913 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:44.746719    2579 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 10 02:14:44.746967 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:44.746927    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wgkcm"
Jul 10 02:14:44.869259 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:44.869195    2579 generic.go:356] "Generic (PLEG): container finished" podID="d64fac57-3910-4a19-8308-f56df2a18b8f" containerID="f125a49c6692dfefed1e4c44caef19d90c5ddefa7694ebc823d3caaf88f67c1c" exitCode=0
Jul 10 02:14:44.869346 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:44.869273    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m42l5" event={"ID":"d64fac57-3910-4a19-8308-f56df2a18b8f","Type":"ContainerDied","Data":"f125a49c6692dfefed1e4c44caef19d90c5ddefa7694ebc823d3caaf88f67c1c"}
Jul 10 02:14:45.561522 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:45.561486    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:45.561674 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:45.561492    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:45.561674 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:45.561625    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:45.561805 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:45.561492    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:45.561805 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:45.561694    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:45.561805 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:45.561744    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:46.179962 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:46.179893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs\") pod \"network-metrics-daemon-dcc8z\" (UID: \"fafe8b6d-e09b-4dac-9687-43f5c7265be7\") " pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:46.180385 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:46.180050    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:46.180385 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:46.180113    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs podName:fafe8b6d-e09b-4dac-9687-43f5c7265be7 nodeName:}" failed. No retries permitted until 2026-07-10 02:15:18.180098437 +0000 UTC m=+65.162856101 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs") pod "network-metrics-daemon-dcc8z" (UID: "fafe8b6d-e09b-4dac-9687-43f5c7265be7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:14:46.280643 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:46.280604    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6f86\" (UniqueName: \"kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86\") pod \"network-check-target-frlf4\" (UID: \"49e0bcec-546d-488b-8fa5-55a8d7589aca\") " pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:46.280818 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:46.280655    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:46.280818 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:46.280762    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:46.280818 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:46.280763    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:14:46.280818 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:46.280786    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:14:46.280818 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:46.280797    2579 projected.go:196] Error preparing data for projected volume kube-api-access-q6f86 for pod openshift-network-diagnostics/network-check-target-frlf4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:46.280818 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:46.280819    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret podName:95280b80-3f30-46c4-be9b-33ff86d5ef24 nodeName:}" failed. No retries permitted until 2026-07-10 02:15:18.280802394 +0000 UTC m=+65.263560252 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret") pod "global-pull-secret-syncer-w59q2" (UID: "95280b80-3f30-46c4-be9b-33ff86d5ef24") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:14:46.281132 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:46.280838    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86 podName:49e0bcec-546d-488b-8fa5-55a8d7589aca nodeName:}" failed. No retries permitted until 2026-07-10 02:15:18.28082667 +0000 UTC m=+65.263584334 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-q6f86" (UniqueName: "kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86") pod "network-check-target-frlf4" (UID: "49e0bcec-546d-488b-8fa5-55a8d7589aca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:14:46.875929 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:46.875848    2579 generic.go:356] "Generic (PLEG): container finished" podID="d64fac57-3910-4a19-8308-f56df2a18b8f" containerID="8900940e3598a84a6c876cc4d5232490e7e89b95ba3de3eba892329f7d663dfe" exitCode=0
Jul 10 02:14:46.875929 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:46.875895    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m42l5" event={"ID":"d64fac57-3910-4a19-8308-f56df2a18b8f","Type":"ContainerDied","Data":"8900940e3598a84a6c876cc4d5232490e7e89b95ba3de3eba892329f7d663dfe"}
Jul 10 02:14:47.561093 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:47.561045    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:47.561762 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:47.561059    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:47.561762 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:47.561171    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:47.561762 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:47.561166    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:47.561762 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:47.561268    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:47.561762 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:47.561344    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:48.882049 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:48.882016    2579 generic.go:356] "Generic (PLEG): container finished" podID="d64fac57-3910-4a19-8308-f56df2a18b8f" containerID="a2c3fb0811fd705744c7b6f85a33d2ef0b15c8dfb9aa9cbe6d6fed2ec0d9cae8" exitCode=0
Jul 10 02:14:48.882411 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:48.882079    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m42l5" event={"ID":"d64fac57-3910-4a19-8308-f56df2a18b8f","Type":"ContainerDied","Data":"a2c3fb0811fd705744c7b6f85a33d2ef0b15c8dfb9aa9cbe6d6fed2ec0d9cae8"}
Jul 10 02:14:49.561186 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:49.561160    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:49.561402 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:49.561161    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:49.561402 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:49.561293    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frlf4" podUID="49e0bcec-546d-488b-8fa5-55a8d7589aca"
Jul 10 02:14:49.561402 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:49.561170    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:49.561402 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:49.561347    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dcc8z" podUID="fafe8b6d-e09b-4dac-9687-43f5c7265be7"
Jul 10 02:14:49.561578 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:49.561433    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w59q2" podUID="95280b80-3f30-46c4-be9b-33ff86d5ef24"
Jul 10 02:14:50.377377 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.377304    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-62.ec2.internal" event="NodeReady"
Jul 10 02:14:50.377917 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.377436    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 10 02:14:50.443325 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.443290    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b56fb775-gjfpv"]
Jul 10 02:14:50.462835 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.462801    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.465719 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.465693    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qj2g2"]
Jul 10 02:14:50.466464 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.466345    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 10 02:14:50.466464 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.466368    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 10 02:14:50.466665 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.466468    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 10 02:14:50.466722 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.466709    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nntqw\""
Jul 10 02:14:50.482687 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.482664    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 10 02:14:50.486870 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.486848    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-f7sbz"]
Jul 10 02:14:50.486999 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.486945    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qj2g2"
Jul 10 02:14:50.490945 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.490897    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 10 02:14:50.491055 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.490978    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 10 02:14:50.491240 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.491221    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2cc4f\""
Jul 10 02:14:50.491484 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.491462    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 10 02:14:50.510975 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.510942    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b56fb775-gjfpv"]
Jul 10 02:14:50.511086 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.510984    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qj2g2"]
Jul 10 02:14:50.511086 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.511004    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f7sbz"]
Jul 10 02:14:50.511086 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.511048    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:50.513950 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.513895    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zcw54\""
Jul 10 02:14:50.514163 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.514126    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 10 02:14:50.514163 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.514142    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 10 02:14:50.614828 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.614794    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-bound-sa-token\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.614828 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.614826    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:50.615010 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.614843    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx9tw\" (UniqueName: \"kubernetes.io/projected/5b633a59-0e3c-45eb-990d-5a55827eefe1-kube-api-access-mx9tw\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:50.615010 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.614888    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/939f5597-cddf-45a4-80b0-d4392803584f-image-registry-private-configuration\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.615010 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.614958    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgtz2\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-kube-api-access-dgtz2\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.615010 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.614980    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5b633a59-0e3c-45eb-990d-5a55827eefe1-config-volume\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:50.615010 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.615004    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/939f5597-cddf-45a4-80b0-d4392803584f-registry-certificates\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.615158 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.615037    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/939f5597-cddf-45a4-80b0-d4392803584f-ca-trust-extracted\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.615158 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.615061    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/939f5597-cddf-45a4-80b0-d4392803584f-installation-pull-secrets\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.615158 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.615088    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b633a59-0e3c-45eb-990d-5a55827eefe1-tmp-dir\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:50.615158 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.615105    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffp9n\" (UniqueName: \"kubernetes.io/projected/6b753a26-89e1-4b05-8a70-d922cf975723-kube-api-access-ffp9n\") pod \"ingress-canary-qj2g2\" (UID: \"6b753a26-89e1-4b05-8a70-d922cf975723\") " pod="openshift-ingress-canary/ingress-canary-qj2g2"
Jul 10 02:14:50.615158 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.615123    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.615301 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.615173    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/939f5597-cddf-45a4-80b0-d4392803584f-trusted-ca\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.615301 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.615187    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b753a26-89e1-4b05-8a70-d922cf975723-cert\") pod \"ingress-canary-qj2g2\" (UID: \"6b753a26-89e1-4b05-8a70-d922cf975723\") " pod="openshift-ingress-canary/ingress-canary-qj2g2"
Jul 10 02:14:50.711210 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.711130    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vg6lr"]
Jul 10 02:14:50.715567 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.715535    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/939f5597-cddf-45a4-80b0-d4392803584f-image-registry-private-configuration\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.715729 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.715582    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgtz2\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-kube-api-access-dgtz2\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.715729 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.715603    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5b633a59-0e3c-45eb-990d-5a55827eefe1-config-volume\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:50.715729 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.715637    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/939f5597-cddf-45a4-80b0-d4392803584f-registry-certificates\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.715729 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.715688    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/939f5597-cddf-45a4-80b0-d4392803584f-ca-trust-extracted\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.715729 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.715713    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/939f5597-cddf-45a4-80b0-d4392803584f-installation-pull-secrets\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.715984 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.715740    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b633a59-0e3c-45eb-990d-5a55827eefe1-tmp-dir\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:50.715984 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.715767    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffp9n\" (UniqueName: \"kubernetes.io/projected/6b753a26-89e1-4b05-8a70-d922cf975723-kube-api-access-ffp9n\") pod \"ingress-canary-qj2g2\" (UID: \"6b753a26-89e1-4b05-8a70-d922cf975723\") " pod="openshift-ingress-canary/ingress-canary-qj2g2"
Jul 10 02:14:50.715984 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.715796    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.715984 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.715844    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/939f5597-cddf-45a4-80b0-d4392803584f-trusted-ca\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.715984 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.715865    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b753a26-89e1-4b05-8a70-d922cf975723-cert\") pod \"ingress-canary-qj2g2\" (UID: \"6b753a26-89e1-4b05-8a70-d922cf975723\") " pod="openshift-ingress-canary/ingress-canary-qj2g2"
Jul 10 02:14:50.715984 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.715907    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-bound-sa-token\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.715984 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.715930    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:50.715984 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.715966    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx9tw\" (UniqueName: \"kubernetes.io/projected/5b633a59-0e3c-45eb-990d-5a55827eefe1-kube-api-access-mx9tw\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:50.716337 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.716178    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b633a59-0e3c-45eb-990d-5a55827eefe1-tmp-dir\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:50.716337 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.716179    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5b633a59-0e3c-45eb-990d-5a55827eefe1-config-volume\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:50.716337 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:50.716316    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:14:50.716623 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:50.716607    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:14:50.716623 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:50.716623    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b56fb775-gjfpv: secret "image-registry-tls" not found
Jul 10 02:14:50.716773 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:50.716666    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls podName:939f5597-cddf-45a4-80b0-d4392803584f nodeName:}" failed. No retries permitted until 2026-07-10 02:14:51.216650358 +0000 UTC m=+38.199408036 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls") pod "image-registry-7b56fb775-gjfpv" (UID: "939f5597-cddf-45a4-80b0-d4392803584f") : secret "image-registry-tls" not found
Jul 10 02:14:50.716773 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:50.716688    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls podName:5b633a59-0e3c-45eb-990d-5a55827eefe1 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:51.216680344 +0000 UTC m=+38.199438022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls") pod "dns-default-f7sbz" (UID: "5b633a59-0e3c-45eb-990d-5a55827eefe1") : secret "dns-default-metrics-tls" not found
Jul 10 02:14:50.716908 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.716787    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/939f5597-cddf-45a4-80b0-d4392803584f-registry-certificates\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.716962 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.716909    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/939f5597-cddf-45a4-80b0-d4392803584f-ca-trust-extracted\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.717028 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.717002    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/939f5597-cddf-45a4-80b0-d4392803584f-trusted-ca\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.720614 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.720569    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/939f5597-cddf-45a4-80b0-d4392803584f-image-registry-private-configuration\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.720721 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.720615    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b753a26-89e1-4b05-8a70-d922cf975723-cert\") pod \"ingress-canary-qj2g2\" (UID: \"6b753a26-89e1-4b05-8a70-d922cf975723\") " pod="openshift-ingress-canary/ingress-canary-qj2g2"
Jul 10 02:14:50.720721 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.720646    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/939f5597-cddf-45a4-80b0-d4392803584f-installation-pull-secrets\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.729450 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.729426    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgtz2\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-kube-api-access-dgtz2\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.729643 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.729524    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-bound-sa-token\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:50.731417 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.731392    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx9tw\" (UniqueName: \"kubernetes.io/projected/5b633a59-0e3c-45eb-990d-5a55827eefe1-kube-api-access-mx9tw\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:50.737359 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.737333    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffp9n\" (UniqueName: \"kubernetes.io/projected/6b753a26-89e1-4b05-8a70-d922cf975723-kube-api-access-ffp9n\") pod \"ingress-canary-qj2g2\" (UID: \"6b753a26-89e1-4b05-8a70-d922cf975723\") " pod="openshift-ingress-canary/ingress-canary-qj2g2"
Jul 10 02:14:50.741783 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.741600    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vg6lr"]
Jul 10 02:14:50.741783 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.741726    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:50.745498 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.744632    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 10 02:14:50.745498 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.744672    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 10 02:14:50.745498 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.744632    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 10 02:14:50.745498 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.744695    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 10 02:14:50.745498 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.744990    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l6nlg\""
Jul 10 02:14:50.797809 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.797785    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qj2g2"
Jul 10 02:14:50.817175 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.817154    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:50.817302 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.817267    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:50.817302 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.817294    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hrq8\" (UniqueName: \"kubernetes.io/projected/de37ac86-4744-475f-bfac-66c48f109148-kube-api-access-5hrq8\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:50.817400 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.817323    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/de37ac86-4744-475f-bfac-66c48f109148-crio-socket\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:50.817400 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.817345    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/de37ac86-4744-475f-bfac-66c48f109148-data-volume\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:50.918326 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.918169    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:50.918454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.918342    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hrq8\" (UniqueName: \"kubernetes.io/projected/de37ac86-4744-475f-bfac-66c48f109148-kube-api-access-5hrq8\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:50.918454 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:50.918312    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 02:14:50.918454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.918390    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/de37ac86-4744-475f-bfac-66c48f109148-crio-socket\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:50.918454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.918430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/de37ac86-4744-475f-bfac-66c48f109148-data-volume\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:50.918684 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.918476    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:50.918684 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:50.918680    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:14:50.918801 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:50.918760    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls podName:de37ac86-4744-475f-bfac-66c48f109148 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:51.418747039 +0000 UTC m=+38.401504703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vg6lr" (UID: "de37ac86-4744-475f-bfac-66c48f109148") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:14:50.918892 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.918868    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/de37ac86-4744-475f-bfac-66c48f109148-crio-socket\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:50.919099 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.919081    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/de37ac86-4744-475f-bfac-66c48f109148-data-volume\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:50.919162 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:50.919139    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm podName:de37ac86-4744-475f-bfac-66c48f109148 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:51.419123019 +0000 UTC m=+38.401880689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vg6lr" (UID: "de37ac86-4744-475f-bfac-66c48f109148") : configmap "kube-rbac-proxy" not found
Jul 10 02:14:50.929611 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.929394    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hrq8\" (UniqueName: \"kubernetes.io/projected/de37ac86-4744-475f-bfac-66c48f109148-kube-api-access-5hrq8\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:50.997921 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:50.997826    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qj2g2"]
Jul 10 02:14:51.002666 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:51.002601    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b753a26_89e1_4b05_8a70_d922cf975723.slice/crio-8a4e79e788d96d05b6b4ca904b58d5506e43ae7642eefc9a467bf4c5fab34a6e WatchSource:0}: Error finding container 8a4e79e788d96d05b6b4ca904b58d5506e43ae7642eefc9a467bf4c5fab34a6e: Status 404 returned error can't find the container with id 8a4e79e788d96d05b6b4ca904b58d5506e43ae7642eefc9a467bf4c5fab34a6e
Jul 10 02:14:51.220134 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.220104    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:51.220309 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.220205    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:51.220309 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:51.220286    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:14:51.220424 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:51.220345    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:14:51.220424 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:51.220360    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b56fb775-gjfpv: secret "image-registry-tls" not found
Jul 10 02:14:51.220424 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:51.220377    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls podName:5b633a59-0e3c-45eb-990d-5a55827eefe1 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:52.220356099 +0000 UTC m=+39.203113767 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls") pod "dns-default-f7sbz" (UID: "5b633a59-0e3c-45eb-990d-5a55827eefe1") : secret "dns-default-metrics-tls" not found
Jul 10 02:14:51.220424 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:51.220406    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls podName:939f5597-cddf-45a4-80b0-d4392803584f nodeName:}" failed. No retries permitted until 2026-07-10 02:14:52.220395393 +0000 UTC m=+39.203153057 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls") pod "image-registry-7b56fb775-gjfpv" (UID: "939f5597-cddf-45a4-80b0-d4392803584f") : secret "image-registry-tls" not found
Jul 10 02:14:51.238959 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.238930    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tfktk"]
Jul 10 02:14:51.269422 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.269355    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tfktk"]
Jul 10 02:14:51.269555 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.269446    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tfktk"
Jul 10 02:14:51.272109 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.272078    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 10 02:14:51.272109 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.272095    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-w4rsj\""
Jul 10 02:14:51.272357 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.272343    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 10 02:14:51.422401 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.422365    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:51.422848 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.422444    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:51.422848 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.422468    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkbw8\" (UniqueName: \"kubernetes.io/projected/95b3336b-3d04-4991-9c18-a95e7cc1661f-kube-api-access-gkbw8\") pod \"migrator-7f59b54f6c-tfktk\" (UID: \"95b3336b-3d04-4991-9c18-a95e7cc1661f\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tfktk"
Jul 10 02:14:51.422848 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:51.422528    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 02:14:51.422848 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:51.422564    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:14:51.422848 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:51.422589    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm podName:de37ac86-4744-475f-bfac-66c48f109148 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:52.422574467 +0000 UTC m=+39.405332131 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vg6lr" (UID: "de37ac86-4744-475f-bfac-66c48f109148") : configmap "kube-rbac-proxy" not found
Jul 10 02:14:51.422848 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:51.422628    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls podName:de37ac86-4744-475f-bfac-66c48f109148 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:52.422609997 +0000 UTC m=+39.405367661 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vg6lr" (UID: "de37ac86-4744-475f-bfac-66c48f109148") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:14:51.523475 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.523392    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkbw8\" (UniqueName: \"kubernetes.io/projected/95b3336b-3d04-4991-9c18-a95e7cc1661f-kube-api-access-gkbw8\") pod \"migrator-7f59b54f6c-tfktk\" (UID: \"95b3336b-3d04-4991-9c18-a95e7cc1661f\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tfktk"
Jul 10 02:14:51.534034 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.534004    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkbw8\" (UniqueName: \"kubernetes.io/projected/95b3336b-3d04-4991-9c18-a95e7cc1661f-kube-api-access-gkbw8\") pod \"migrator-7f59b54f6c-tfktk\" (UID: \"95b3336b-3d04-4991-9c18-a95e7cc1661f\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tfktk"
Jul 10 02:14:51.563925 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.563898    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:14:51.563925 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.563928    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:14:51.564132 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.563898    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:14:51.566496 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.566476    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 02:14:51.566650 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.566569    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pmctl\""
Jul 10 02:14:51.566650 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.566569    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 02:14:51.566778 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.566759    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 02:14:51.566835 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.566762    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 02:14:51.566835 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.566768    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-htmn4\""
Jul 10 02:14:51.578445 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.578265    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tfktk"
Jul 10 02:14:51.731920 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.731442    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tfktk"]
Jul 10 02:14:51.736744 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:51.736714    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b3336b_3d04_4991_9c18_a95e7cc1661f.slice/crio-25790109b1c63d34b13db712bd5d9efa2e264a437ed0f8f4fc11ed7ecae8760a WatchSource:0}: Error finding container 25790109b1c63d34b13db712bd5d9efa2e264a437ed0f8f4fc11ed7ecae8760a: Status 404 returned error can't find the container with id 25790109b1c63d34b13db712bd5d9efa2e264a437ed0f8f4fc11ed7ecae8760a
Jul 10 02:14:51.890325 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.890288    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qj2g2" event={"ID":"6b753a26-89e1-4b05-8a70-d922cf975723","Type":"ContainerStarted","Data":"8a4e79e788d96d05b6b4ca904b58d5506e43ae7642eefc9a467bf4c5fab34a6e"}
Jul 10 02:14:51.891936 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:51.891894    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tfktk" event={"ID":"95b3336b-3d04-4991-9c18-a95e7cc1661f","Type":"ContainerStarted","Data":"25790109b1c63d34b13db712bd5d9efa2e264a437ed0f8f4fc11ed7ecae8760a"}
Jul 10 02:14:52.085675 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:52.085651    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9gnrj_cdca22f8-36eb-4162-abd9-9fed3a472a58/dns-node-resolver/0.log"
Jul 10 02:14:52.229044 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:52.228972    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:52.229203 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:52.229053    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:52.229203 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:52.229187    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:14:52.229374 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:52.229360    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls podName:5b633a59-0e3c-45eb-990d-5a55827eefe1 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:54.22933724 +0000 UTC m=+41.212094922 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls") pod "dns-default-f7sbz" (UID: "5b633a59-0e3c-45eb-990d-5a55827eefe1") : secret "dns-default-metrics-tls" not found
Jul 10 02:14:52.229829 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:52.229807    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:14:52.229829 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:52.229828    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b56fb775-gjfpv: secret "image-registry-tls" not found
Jul 10 02:14:52.229978 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:52.229873    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls podName:939f5597-cddf-45a4-80b0-d4392803584f nodeName:}" failed. No retries permitted until 2026-07-10 02:14:54.229857773 +0000 UTC m=+41.212615437 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls") pod "image-registry-7b56fb775-gjfpv" (UID: "939f5597-cddf-45a4-80b0-d4392803584f") : secret "image-registry-tls" not found
Jul 10 02:14:52.430626 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:52.430591    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:52.431109 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:52.430691    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:52.431109 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:52.430842    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:14:52.431109 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:52.430910    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls podName:de37ac86-4744-475f-bfac-66c48f109148 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:54.430888264 +0000 UTC m=+41.413646136 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vg6lr" (UID: "de37ac86-4744-475f-bfac-66c48f109148") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:14:52.431256 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:52.431242    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 02:14:52.431302 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:52.431280    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm podName:de37ac86-4744-475f-bfac-66c48f109148 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:54.431269259 +0000 UTC m=+41.414026923 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vg6lr" (UID: "de37ac86-4744-475f-bfac-66c48f109148") : configmap "kube-rbac-proxy" not found
Jul 10 02:14:53.302257 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.302224    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-wnj5f"]
Jul 10 02:14:53.312612 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.312588    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-wnj5f"
Jul 10 02:14:53.318638 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.317018    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-wnj5f"]
Jul 10 02:14:53.319741 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.319621    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 10 02:14:53.319872 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.319823    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 10 02:14:53.319937 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.319895    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 10 02:14:53.320159 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.319895    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 10 02:14:53.320808 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.320627    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-wqhsr\""
Jul 10 02:14:53.440580 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.440542    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6356f1b4-d2ec-4467-a79a-6f4d2d01daf1-signing-cabundle\") pod \"service-ca-79f68b9f4f-wnj5f\" (UID: \"6356f1b4-d2ec-4467-a79a-6f4d2d01daf1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wnj5f"
Jul 10 02:14:53.440580 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.440578    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6356f1b4-d2ec-4467-a79a-6f4d2d01daf1-signing-key\") pod \"service-ca-79f68b9f4f-wnj5f\" (UID: \"6356f1b4-d2ec-4467-a79a-6f4d2d01daf1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wnj5f"
Jul 10 02:14:53.441071 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.440607    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s8fw\" (UniqueName: \"kubernetes.io/projected/6356f1b4-d2ec-4467-a79a-6f4d2d01daf1-kube-api-access-6s8fw\") pod \"service-ca-79f68b9f4f-wnj5f\" (UID: \"6356f1b4-d2ec-4467-a79a-6f4d2d01daf1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wnj5f"
Jul 10 02:14:53.485851 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.485821    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mnqnz_79684caa-c963-4b62-8a74-ed4e87d3f201/node-ca/0.log"
Jul 10 02:14:53.542236 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.542158    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6356f1b4-d2ec-4467-a79a-6f4d2d01daf1-signing-cabundle\") pod \"service-ca-79f68b9f4f-wnj5f\" (UID: \"6356f1b4-d2ec-4467-a79a-6f4d2d01daf1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wnj5f"
Jul 10 02:14:53.542236 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.542191    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6356f1b4-d2ec-4467-a79a-6f4d2d01daf1-signing-key\") pod \"service-ca-79f68b9f4f-wnj5f\" (UID: \"6356f1b4-d2ec-4467-a79a-6f4d2d01daf1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wnj5f"
Jul 10 02:14:53.542236 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.542217    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s8fw\" (UniqueName: \"kubernetes.io/projected/6356f1b4-d2ec-4467-a79a-6f4d2d01daf1-kube-api-access-6s8fw\") pod \"service-ca-79f68b9f4f-wnj5f\" (UID: \"6356f1b4-d2ec-4467-a79a-6f4d2d01daf1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wnj5f"
Jul 10 02:14:53.542959 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.542935    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6356f1b4-d2ec-4467-a79a-6f4d2d01daf1-signing-cabundle\") pod \"service-ca-79f68b9f4f-wnj5f\" (UID: \"6356f1b4-d2ec-4467-a79a-6f4d2d01daf1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wnj5f"
Jul 10 02:14:53.546036 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.546017    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6356f1b4-d2ec-4467-a79a-6f4d2d01daf1-signing-key\") pod \"service-ca-79f68b9f4f-wnj5f\" (UID: \"6356f1b4-d2ec-4467-a79a-6f4d2d01daf1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wnj5f"
Jul 10 02:14:53.558732 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.558705    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s8fw\" (UniqueName: \"kubernetes.io/projected/6356f1b4-d2ec-4467-a79a-6f4d2d01daf1-kube-api-access-6s8fw\") pod \"service-ca-79f68b9f4f-wnj5f\" (UID: \"6356f1b4-d2ec-4467-a79a-6f4d2d01daf1\") " pod="openshift-service-ca/service-ca-79f68b9f4f-wnj5f"
Jul 10 02:14:53.627148 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:53.627113    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-wnj5f"
Jul 10 02:14:54.248686 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:54.248632    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:54.248870 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:54.248742    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:54.248870 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:54.248790    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:14:54.248870 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:54.248807    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b56fb775-gjfpv: secret "image-registry-tls" not found
Jul 10 02:14:54.248870 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:54.248866    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls podName:939f5597-cddf-45a4-80b0-d4392803584f nodeName:}" failed. No retries permitted until 2026-07-10 02:14:58.248846861 +0000 UTC m=+45.231604529 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls") pod "image-registry-7b56fb775-gjfpv" (UID: "939f5597-cddf-45a4-80b0-d4392803584f") : secret "image-registry-tls" not found
Jul 10 02:14:54.249028 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:54.248908    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:14:54.249028 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:54.248960    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls podName:5b633a59-0e3c-45eb-990d-5a55827eefe1 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:58.24894383 +0000 UTC m=+45.231701494 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls") pod "dns-default-f7sbz" (UID: "5b633a59-0e3c-45eb-990d-5a55827eefe1") : secret "dns-default-metrics-tls" not found
Jul 10 02:14:54.450040 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:54.449995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:54.450453 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:54.450082    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:54.450453 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:54.450135    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 02:14:54.450453 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:54.450212    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm podName:de37ac86-4744-475f-bfac-66c48f109148 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:58.450190128 +0000 UTC m=+45.432947792 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vg6lr" (UID: "de37ac86-4744-475f-bfac-66c48f109148") : configmap "kube-rbac-proxy" not found
Jul 10 02:14:54.450453 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:54.450222    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:14:54.450453 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:54.450274    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls podName:de37ac86-4744-475f-bfac-66c48f109148 nodeName:}" failed. No retries permitted until 2026-07-10 02:14:58.450257512 +0000 UTC m=+45.433015196 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vg6lr" (UID: "de37ac86-4744-475f-bfac-66c48f109148") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:14:55.756703 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:55.756682    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-wnj5f"]
Jul 10 02:14:55.758688 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:14:55.758665    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6356f1b4_d2ec_4467_a79a_6f4d2d01daf1.slice/crio-486cf4eaacb455e060859402a1afdbc8d8434f56022aaa5d6baad17f4ec27c8a WatchSource:0}: Error finding container 486cf4eaacb455e060859402a1afdbc8d8434f56022aaa5d6baad17f4ec27c8a: Status 404 returned error can't find the container with id 486cf4eaacb455e060859402a1afdbc8d8434f56022aaa5d6baad17f4ec27c8a
Jul 10 02:14:55.904439 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:55.904276    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tfktk" event={"ID":"95b3336b-3d04-4991-9c18-a95e7cc1661f","Type":"ContainerStarted","Data":"ce1b87069513ca605d338de3d0ea95a931b13b16c747c865a1ba6110b9022f86"}
Jul 10 02:14:55.905993 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:55.905964    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-wnj5f" event={"ID":"6356f1b4-d2ec-4467-a79a-6f4d2d01daf1","Type":"ContainerStarted","Data":"486cf4eaacb455e060859402a1afdbc8d8434f56022aaa5d6baad17f4ec27c8a"}
Jul 10 02:14:55.908908 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:55.908886    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m42l5" event={"ID":"d64fac57-3910-4a19-8308-f56df2a18b8f","Type":"ContainerStarted","Data":"90fb041fbfdf81aa3ac3f47294147cf6213a3d86bdfd8beed187a45e61b0cd05"}
Jul 10 02:14:55.910415 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:55.910386    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qj2g2" event={"ID":"6b753a26-89e1-4b05-8a70-d922cf975723","Type":"ContainerStarted","Data":"5da33ee25ddc0ee2acec9d7c1476a0b1ff91699e73e539bf18bb874da6ea1c1d"}
Jul 10 02:14:55.956288 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:55.956245    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qj2g2" podStartSLOduration=1.362610752 podStartE2EDuration="5.956230032s" totalImagesPullingTime="4.59361928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:14:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:55.955262947 +0000 UTC m=+42.938020633" watchObservedRunningTime="2026-07-10 02:14:55.956230032 +0000 UTC m=+42.938987711"
Jul 10 02:14:56.915745 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:56.915668    2579 generic.go:356] "Generic (PLEG): container finished" podID="d64fac57-3910-4a19-8308-f56df2a18b8f" containerID="90fb041fbfdf81aa3ac3f47294147cf6213a3d86bdfd8beed187a45e61b0cd05" exitCode=0
Jul 10 02:14:56.916192 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:56.915746    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m42l5" event={"ID":"d64fac57-3910-4a19-8308-f56df2a18b8f","Type":"ContainerDied","Data":"90fb041fbfdf81aa3ac3f47294147cf6213a3d86bdfd8beed187a45e61b0cd05"}
Jul 10 02:14:56.918053 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:56.918023    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tfktk" event={"ID":"95b3336b-3d04-4991-9c18-a95e7cc1661f","Type":"ContainerStarted","Data":"09281d2b46fed2ff802aeb6131a6c53d38afcd7b1b7bc1514bfeb20c8efe2150"}
Jul 10 02:14:56.975830 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:56.975782    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-tfktk" podStartSLOduration=2.1156023409999998 podStartE2EDuration="5.975762351s" totalImagesPullingTime="3.86016001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:14:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:56.975701949 +0000 UTC m=+43.958459636" watchObservedRunningTime="2026-07-10 02:14:56.975762351 +0000 UTC m=+43.958520040"
Jul 10 02:14:57.924168 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:57.924132    2579 generic.go:356] "Generic (PLEG): container finished" podID="d64fac57-3910-4a19-8308-f56df2a18b8f" containerID="ec4cd493f5621093ec08321ab3bd0d0dc7da7e59a6f9dddd3710ace387fb5514" exitCode=0
Jul 10 02:14:57.924591 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:57.924209    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m42l5" event={"ID":"d64fac57-3910-4a19-8308-f56df2a18b8f","Type":"ContainerDied","Data":"ec4cd493f5621093ec08321ab3bd0d0dc7da7e59a6f9dddd3710ace387fb5514"}
Jul 10 02:14:58.286651 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:58.286628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:14:58.286772 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:58.286714    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:14:58.286830 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:58.286770    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:14:58.286879 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:58.286836    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls podName:5b633a59-0e3c-45eb-990d-5a55827eefe1 nodeName:}" failed. No retries permitted until 2026-07-10 02:15:06.286819252 +0000 UTC m=+53.269576916 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls") pod "dns-default-f7sbz" (UID: "5b633a59-0e3c-45eb-990d-5a55827eefe1") : secret "dns-default-metrics-tls" not found
Jul 10 02:14:58.286879 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:58.286858    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:14:58.286879 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:58.286877    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b56fb775-gjfpv: secret "image-registry-tls" not found
Jul 10 02:14:58.287005 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:58.286935    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls podName:939f5597-cddf-45a4-80b0-d4392803584f nodeName:}" failed. No retries permitted until 2026-07-10 02:15:06.286917869 +0000 UTC m=+53.269675551 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls") pod "image-registry-7b56fb775-gjfpv" (UID: "939f5597-cddf-45a4-80b0-d4392803584f") : secret "image-registry-tls" not found
Jul 10 02:14:58.488704 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:58.488622    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:58.488870 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:58.488730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:14:58.488870 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:58.488758    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:14:58.488870 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:58.488814    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 02:14:58.488870 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:58.488820    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls podName:de37ac86-4744-475f-bfac-66c48f109148 nodeName:}" failed. No retries permitted until 2026-07-10 02:15:06.488804934 +0000 UTC m=+53.471562599 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vg6lr" (UID: "de37ac86-4744-475f-bfac-66c48f109148") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:14:58.488870 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:14:58.488861    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm podName:de37ac86-4744-475f-bfac-66c48f109148 nodeName:}" failed. No retries permitted until 2026-07-10 02:15:06.488846718 +0000 UTC m=+53.471604396 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vg6lr" (UID: "de37ac86-4744-475f-bfac-66c48f109148") : configmap "kube-rbac-proxy" not found
Jul 10 02:14:58.928636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:58.928599    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-wnj5f" event={"ID":"6356f1b4-d2ec-4467-a79a-6f4d2d01daf1","Type":"ContainerStarted","Data":"bc031cdbbb7eb8c8029e1352b1c067b05fe0f3b698cdc2d2c31a18310a389909"}
Jul 10 02:14:58.935583 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:58.935552    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-m42l5" event={"ID":"d64fac57-3910-4a19-8308-f56df2a18b8f","Type":"ContainerStarted","Data":"e32a2877ac7ce940a043b6ff95c4928bb9d65d6be60c06a7eaf354cb33b74b64"}
Jul 10 02:14:58.951293 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:58.951252    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-wnj5f" podStartSLOduration=3.199260586 podStartE2EDuration="5.951237672s" totalImagesPullingTime="2.751977086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:14:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:58.950341633 +0000 UTC m=+45.933099320" watchObservedRunningTime="2026-07-10 02:14:58.951237672 +0000 UTC m=+45.933995358"
Jul 10 02:14:58.981166 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:14:58.981113    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-m42l5" podStartSLOduration=10.123257681 podStartE2EDuration="44.98109518s" totalImagesPullingTime="34.857837499s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:14:58.979104803 +0000 UTC m=+45.961862488" watchObservedRunningTime="2026-07-10 02:14:58.98109518 +0000 UTC m=+45.963852866"
Jul 10 02:15:06.350309 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:06.350267    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:15:06.350947 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:06.350390    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:15:06.353028 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:06.353001    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls\") pod \"image-registry-7b56fb775-gjfpv\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") " pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:15:06.353197 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:06.353177    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b633a59-0e3c-45eb-990d-5a55827eefe1-metrics-tls\") pod \"dns-default-f7sbz\" (UID: \"5b633a59-0e3c-45eb-990d-5a55827eefe1\") " pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:15:06.375201 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:06.375159    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:15:06.420655 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:06.420617    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:15:06.526450 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:06.526374    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b56fb775-gjfpv"]
Jul 10 02:15:06.532391 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:15:06.532348    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod939f5597_cddf_45a4_80b0_d4392803584f.slice/crio-fad18897d3853bf92b92593c3110349b069573855102e75f597d07854800ec07 WatchSource:0}: Error finding container fad18897d3853bf92b92593c3110349b069573855102e75f597d07854800ec07: Status 404 returned error can't find the container with id fad18897d3853bf92b92593c3110349b069573855102e75f597d07854800ec07
Jul 10 02:15:06.551935 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:06.551904    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:15:06.552063 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:06.552023    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:15:06.552271 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:15:06.552243    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:15:06.552374 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:15:06.552318    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls podName:de37ac86-4744-475f-bfac-66c48f109148 nodeName:}" failed. No retries permitted until 2026-07-10 02:15:22.552297636 +0000 UTC m=+69.535055323 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vg6lr" (UID: "de37ac86-4744-475f-bfac-66c48f109148") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:15:06.552548 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:06.552524    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/de37ac86-4744-475f-bfac-66c48f109148-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:15:06.576801 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:06.576776    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f7sbz"]
Jul 10 02:15:06.580238 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:15:06.580214    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b633a59_0e3c_45eb_990d_5a55827eefe1.slice/crio-3b8bde0b3f972a51753d3cf554d0037a16c219d82341d6d8fa08b5e88837cd13 WatchSource:0}: Error finding container 3b8bde0b3f972a51753d3cf554d0037a16c219d82341d6d8fa08b5e88837cd13: Status 404 returned error can't find the container with id 3b8bde0b3f972a51753d3cf554d0037a16c219d82341d6d8fa08b5e88837cd13
Jul 10 02:15:06.957318 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:06.957279    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f7sbz" event={"ID":"5b633a59-0e3c-45eb-990d-5a55827eefe1","Type":"ContainerStarted","Data":"3b8bde0b3f972a51753d3cf554d0037a16c219d82341d6d8fa08b5e88837cd13"}
Jul 10 02:15:06.958796 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:06.958774    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b56fb775-gjfpv" event={"ID":"939f5597-cddf-45a4-80b0-d4392803584f","Type":"ContainerStarted","Data":"8fe7c07d2417748a04fd744f6f116ef8479fa789fad1e9cdf58b31754a18faf4"}
Jul 10 02:15:06.958917 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:06.958814    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b56fb775-gjfpv" event={"ID":"939f5597-cddf-45a4-80b0-d4392803584f","Type":"ContainerStarted","Data":"fad18897d3853bf92b92593c3110349b069573855102e75f597d07854800ec07"}
Jul 10 02:15:06.958917 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:06.958912    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:15:08.967300 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:08.967270    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f7sbz" event={"ID":"5b633a59-0e3c-45eb-990d-5a55827eefe1","Type":"ContainerStarted","Data":"a0b5973ea2b5ed7660a1e3c1a86ee0ce1bf24621a9a047189432dffc53f3d937"}
Jul 10 02:15:08.967731 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:08.967343    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f7sbz" event={"ID":"5b633a59-0e3c-45eb-990d-5a55827eefe1","Type":"ContainerStarted","Data":"05993fced5bd8737ef39d693506cb96bcb45e870e62b2fa25c1bd9196720f84b"}
Jul 10 02:15:08.967731 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:08.967377    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:15:08.990085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:08.990037    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b56fb775-gjfpv" podStartSLOduration=39.990020088 podStartE2EDuration="39.990020088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:14:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:06.979952196 +0000 UTC m=+53.962709882" watchObservedRunningTime="2026-07-10 02:15:08.990020088 +0000 UTC m=+55.972777776"
Jul 10 02:15:08.990242 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:08.990159    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-f7sbz" podStartSLOduration=17.10065752 podStartE2EDuration="18.990149425s" totalImagesPullingTime="1.889491905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:14:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:08.989726432 +0000 UTC m=+55.972484118" watchObservedRunningTime="2026-07-10 02:15:08.990149425 +0000 UTC m=+55.972907113"
Jul 10 02:15:13.273737 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.273706    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-7fqxc"]
Jul 10 02:15:13.278534 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.278517    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-7fqxc"
Jul 10 02:15:13.282422 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.282402    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6864ccb968-9dcnh"]
Jul 10 02:15:13.283399 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.283379    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 10 02:15:13.284536 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.284516    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9wr42\""
Jul 10 02:15:13.284645 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.284521    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 10 02:15:13.285193 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.285174    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6864ccb968-9dcnh"
Jul 10 02:15:13.300581 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:15:13.300553    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"open-cluster-management-image-pull-credentials\" is forbidden: User \"system:node:ip-10-0-132-62.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-132-62.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" type="*v1.Secret"
Jul 10 02:15:13.300664 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:15:13.300553    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"managed-serviceaccount-dockercfg-mpssq\" is forbidden: User \"system:node:ip-10-0-132-62.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-132-62.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-mpssq\"" type="*v1.Secret"
Jul 10 02:15:13.300868 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:15:13.300846    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"managed-serviceaccount-hub-kubeconfig\" is forbidden: User \"system:node:ip-10-0-132-62.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-132-62.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" type="*v1.Secret"
Jul 10 02:15:13.300921 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.300907    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 10 02:15:13.301081 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.301068    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a07b77b4-3144-4cbb-8937-0a6bd23da2c0-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7fqxc\" (UID: \"a07b77b4-3144-4cbb-8937-0a6bd23da2c0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7fqxc"
Jul 10 02:15:13.301113 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.301091    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll5cl\" (UniqueName: \"kubernetes.io/projected/9b7250de-fd0e-4f05-9557-9a96d2871b37-kube-api-access-ll5cl\") pod \"managed-serviceaccount-addon-agent-6864ccb968-9dcnh\" (UID: \"9b7250de-fd0e-4f05-9557-9a96d2871b37\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6864ccb968-9dcnh"
Jul 10 02:15:13.301145 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.301135    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b7250de-fd0e-4f05-9557-9a96d2871b37-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6864ccb968-9dcnh\" (UID: \"9b7250de-fd0e-4f05-9557-9a96d2871b37\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6864ccb968-9dcnh"
Jul 10 02:15:13.301180 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.301170    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a07b77b4-3144-4cbb-8937-0a6bd23da2c0-nginx-conf\") pod \"networking-console-plugin-6cf856758c-7fqxc\" (UID: \"a07b77b4-3144-4cbb-8937-0a6bd23da2c0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7fqxc"
Jul 10 02:15:13.311681 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.311661    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"]
Jul 10 02:15:13.312698 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.312684    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 10 02:15:13.314608 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.314593    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-7fqxc"]
Jul 10 02:15:13.314682 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.314672    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.322086 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.322061    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 10 02:15:13.322280 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.322266    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 10 02:15:13.322424 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.322409    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 10 02:15:13.323334 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.323319    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 10 02:15:13.326621 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.326602    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6864ccb968-9dcnh"]
Jul 10 02:15:13.359381 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.359350    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"]
Jul 10 02:15:13.401653 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.401624    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3444d515-1f71-4a24-a1fb-ee8ea3895e28-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.401653 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.401652    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjsj9\" (UniqueName: \"kubernetes.io/projected/3444d515-1f71-4a24-a1fb-ee8ea3895e28-kube-api-access-tjsj9\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.401868 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.401672    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3444d515-1f71-4a24-a1fb-ee8ea3895e28-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.401868 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.401691    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a07b77b4-3144-4cbb-8937-0a6bd23da2c0-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7fqxc\" (UID: \"a07b77b4-3144-4cbb-8937-0a6bd23da2c0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7fqxc"
Jul 10 02:15:13.401868 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.401744    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll5cl\" (UniqueName: \"kubernetes.io/projected/9b7250de-fd0e-4f05-9557-9a96d2871b37-kube-api-access-ll5cl\") pod \"managed-serviceaccount-addon-agent-6864ccb968-9dcnh\" (UID: \"9b7250de-fd0e-4f05-9557-9a96d2871b37\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6864ccb968-9dcnh"
Jul 10 02:15:13.401868 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.401771    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3444d515-1f71-4a24-a1fb-ee8ea3895e28-ca\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.401868 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.401826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b7250de-fd0e-4f05-9557-9a96d2871b37-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6864ccb968-9dcnh\" (UID: \"9b7250de-fd0e-4f05-9557-9a96d2871b37\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6864ccb968-9dcnh"
Jul 10 02:15:13.401868 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.401849    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a07b77b4-3144-4cbb-8937-0a6bd23da2c0-nginx-conf\") pod \"networking-console-plugin-6cf856758c-7fqxc\" (UID: \"a07b77b4-3144-4cbb-8937-0a6bd23da2c0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7fqxc"
Jul 10 02:15:13.401868 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.401867    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3444d515-1f71-4a24-a1fb-ee8ea3895e28-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.402154 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.401889    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3444d515-1f71-4a24-a1fb-ee8ea3895e28-hub\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.406402 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.406383    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 10 02:15:13.406402 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.406391    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 10 02:15:13.413224 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.413155    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a07b77b4-3144-4cbb-8937-0a6bd23da2c0-nginx-conf\") pod \"networking-console-plugin-6cf856758c-7fqxc\" (UID: \"a07b77b4-3144-4cbb-8937-0a6bd23da2c0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7fqxc"
Jul 10 02:15:13.414953 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.414935    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a07b77b4-3144-4cbb-8937-0a6bd23da2c0-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-7fqxc\" (UID: \"a07b77b4-3144-4cbb-8937-0a6bd23da2c0\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-7fqxc"
Jul 10 02:15:13.443152 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.443129    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b56fb775-gjfpv"]
Jul 10 02:15:13.447080 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.447062    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 10 02:15:13.456690 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.456670    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 10 02:15:13.469674 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.469651    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll5cl\" (UniqueName: \"kubernetes.io/projected/9b7250de-fd0e-4f05-9557-9a96d2871b37-kube-api-access-ll5cl\") pod \"managed-serviceaccount-addon-agent-6864ccb968-9dcnh\" (UID: \"9b7250de-fd0e-4f05-9557-9a96d2871b37\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6864ccb968-9dcnh"
Jul 10 02:15:13.491236 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.491208    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"]
Jul 10 02:15:13.495611 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.495593    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.502213 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.502187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3444d515-1f71-4a24-a1fb-ee8ea3895e28-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.502313 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.502216    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3444d515-1f71-4a24-a1fb-ee8ea3895e28-hub\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.502313 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.502267    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3444d515-1f71-4a24-a1fb-ee8ea3895e28-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.502313 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.502286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjsj9\" (UniqueName: \"kubernetes.io/projected/3444d515-1f71-4a24-a1fb-ee8ea3895e28-kube-api-access-tjsj9\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.502313 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.502302    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3444d515-1f71-4a24-a1fb-ee8ea3895e28-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.502545 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.502341    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3444d515-1f71-4a24-a1fb-ee8ea3895e28-ca\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.502863 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.502841    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3444d515-1f71-4a24-a1fb-ee8ea3895e28-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.506332 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.506310    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 10 02:15:13.506602 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.506585    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 10 02:15:13.506706 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.506622    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 10 02:15:13.506706 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.506633    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 10 02:15:13.507591 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.507571    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"]
Jul 10 02:15:13.515822 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.515795    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3444d515-1f71-4a24-a1fb-ee8ea3895e28-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.515908 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.515807    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3444d515-1f71-4a24-a1fb-ee8ea3895e28-hub\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.515908 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.515889    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3444d515-1f71-4a24-a1fb-ee8ea3895e28-ca\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.516042 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.516025    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3444d515-1f71-4a24-a1fb-ee8ea3895e28-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.519600 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.519579    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjsj9\" (UniqueName: \"kubernetes.io/projected/3444d515-1f71-4a24-a1fb-ee8ea3895e28-kube-api-access-tjsj9\") pod \"cluster-proxy-proxy-agent-7b4b8477f9-hnpt7\" (UID: \"3444d515-1f71-4a24-a1fb-ee8ea3895e28\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:13.589535 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.589452    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9wr42\""
Jul 10 02:15:13.597788 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.597760    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-7fqxc"
Jul 10 02:15:13.602972 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.602954    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90897a69-5317-40c8-9536-1221271d7623-registry-tls\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.603028 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.602980    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5c2l\" (UniqueName: \"kubernetes.io/projected/90897a69-5317-40c8-9536-1221271d7623-kube-api-access-g5c2l\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.603028 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.602999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90897a69-5317-40c8-9536-1221271d7623-image-registry-private-configuration\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.603095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.603056    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90897a69-5317-40c8-9536-1221271d7623-installation-pull-secrets\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.603127 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.603113    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90897a69-5317-40c8-9536-1221271d7623-registry-certificates\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.603172 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.603156    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90897a69-5317-40c8-9536-1221271d7623-trusted-ca\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.603210 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.603190    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90897a69-5317-40c8-9536-1221271d7623-ca-trust-extracted\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.603245 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.603209    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90897a69-5317-40c8-9536-1221271d7623-bound-sa-token\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.703978 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.703950    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90897a69-5317-40c8-9536-1221271d7623-ca-trust-extracted\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.703978 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.703984    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90897a69-5317-40c8-9536-1221271d7623-bound-sa-token\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.704191 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.704059    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90897a69-5317-40c8-9536-1221271d7623-registry-tls\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.704191 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.704086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5c2l\" (UniqueName: \"kubernetes.io/projected/90897a69-5317-40c8-9536-1221271d7623-kube-api-access-g5c2l\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.704191 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.704116    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90897a69-5317-40c8-9536-1221271d7623-image-registry-private-configuration\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.704191 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.704157    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90897a69-5317-40c8-9536-1221271d7623-installation-pull-secrets\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.704392 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.704223    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90897a69-5317-40c8-9536-1221271d7623-registry-certificates\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.704392 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.704313    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90897a69-5317-40c8-9536-1221271d7623-trusted-ca\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.704392 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.704358    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/90897a69-5317-40c8-9536-1221271d7623-ca-trust-extracted\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.705204 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.705179    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/90897a69-5317-40c8-9536-1221271d7623-registry-certificates\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.705304 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.705180    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/90897a69-5317-40c8-9536-1221271d7623-trusted-ca\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.706563 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.706543    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/90897a69-5317-40c8-9536-1221271d7623-installation-pull-secrets\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.706635 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.706587    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/90897a69-5317-40c8-9536-1221271d7623-registry-tls\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.706635 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.706620    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/90897a69-5317-40c8-9536-1221271d7623-image-registry-private-configuration\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.718464 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.718446    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90897a69-5317-40c8-9536-1221271d7623-bound-sa-token\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.718597 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.718581    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5c2l\" (UniqueName: \"kubernetes.io/projected/90897a69-5317-40c8-9536-1221271d7623-kube-api-access-g5c2l\") pod \"image-registry-6f7d7fccf8-ps2l5\" (UID: \"90897a69-5317-40c8-9536-1221271d7623\") " pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.728803 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.728782    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-7fqxc"]
Jul 10 02:15:13.731369 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:15:13.731348    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda07b77b4_3144_4cbb_8937_0a6bd23da2c0.slice/crio-b7cade2dd3b32bd22d204f1c0d042f58ed9a51fba1d12d7fa092169ab9c2fe9c WatchSource:0}: Error finding container b7cade2dd3b32bd22d204f1c0d042f58ed9a51fba1d12d7fa092169ab9c2fe9c: Status 404 returned error can't find the container with id b7cade2dd3b32bd22d204f1c0d042f58ed9a51fba1d12d7fa092169ab9c2fe9c
Jul 10 02:15:13.803567 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.803533    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:13.876820 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.876788    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t2bng"
Jul 10 02:15:13.945982 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.945953    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"]
Jul 10 02:15:13.948820 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:15:13.948788    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90897a69_5317_40c8_9536_1221271d7623.slice/crio-3499c4c5eede370a482a2a271afdd42d7e7a36870796e0fd7c39b43da81395aa WatchSource:0}: Error finding container 3499c4c5eede370a482a2a271afdd42d7e7a36870796e0fd7c39b43da81395aa: Status 404 returned error can't find the container with id 3499c4c5eede370a482a2a271afdd42d7e7a36870796e0fd7c39b43da81395aa
Jul 10 02:15:13.985973 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.985941    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5" event={"ID":"90897a69-5317-40c8-9536-1221271d7623","Type":"ContainerStarted","Data":"3499c4c5eede370a482a2a271afdd42d7e7a36870796e0fd7c39b43da81395aa"}
Jul 10 02:15:13.987152 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:13.987129    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-7fqxc" event={"ID":"a07b77b4-3144-4cbb-8937-0a6bd23da2c0","Type":"ContainerStarted","Data":"b7cade2dd3b32bd22d204f1c0d042f58ed9a51fba1d12d7fa092169ab9c2fe9c"}
Jul 10 02:15:14.281405 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:14.281337    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 10 02:15:14.285066 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:14.285044    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b7250de-fd0e-4f05-9557-9a96d2871b37-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6864ccb968-9dcnh\" (UID: \"9b7250de-fd0e-4f05-9557-9a96d2871b37\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6864ccb968-9dcnh"
Jul 10 02:15:14.298458 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:14.298434    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 10 02:15:14.302787 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:14.302759    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"
Jul 10 02:15:14.464850 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:14.464748    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7"]
Jul 10 02:15:14.468536 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:15:14.468106    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3444d515_1f71_4a24_a1fb_ee8ea3895e28.slice/crio-d2f2a3d63d7ed19eb21b210822a57f61762a8b730dbb7ca4891e8e5bba169287 WatchSource:0}: Error finding container d2f2a3d63d7ed19eb21b210822a57f61762a8b730dbb7ca4891e8e5bba169287: Status 404 returned error can't find the container with id d2f2a3d63d7ed19eb21b210822a57f61762a8b730dbb7ca4891e8e5bba169287
Jul 10 02:15:14.741040 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:14.741007    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-mpssq\""
Jul 10 02:15:14.747311 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:14.747283    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6864ccb968-9dcnh"
Jul 10 02:15:14.992317 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:14.992226    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5" event={"ID":"90897a69-5317-40c8-9536-1221271d7623","Type":"ContainerStarted","Data":"c9b39c865db2eb9627d7f2359bb05361796ba07f5eb8546b829042b44f472733"}
Jul 10 02:15:14.992468 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:14.992359    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:14.993451 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:14.993427    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7" event={"ID":"3444d515-1f71-4a24-a1fb-ee8ea3895e28","Type":"ContainerStarted","Data":"d2f2a3d63d7ed19eb21b210822a57f61762a8b730dbb7ca4891e8e5bba169287"}
Jul 10 02:15:15.014426 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:15.014402    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6864ccb968-9dcnh"]
Jul 10 02:15:15.018020 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:15:15.017996    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b7250de_fd0e_4f05_9557_9a96d2871b37.slice/crio-ea51c3464cb8a77b3c86e8c6197b4e581be8f668c9cfc9f6cdfed262960ad97d WatchSource:0}: Error finding container ea51c3464cb8a77b3c86e8c6197b4e581be8f668c9cfc9f6cdfed262960ad97d: Status 404 returned error can't find the container with id ea51c3464cb8a77b3c86e8c6197b4e581be8f668c9cfc9f6cdfed262960ad97d
Jul 10 02:15:16.000486 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:15.999567    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-7fqxc" event={"ID":"a07b77b4-3144-4cbb-8937-0a6bd23da2c0","Type":"ContainerStarted","Data":"65736fa227250fc930456d2987e06da0514b24b34678f5b6f3fbb4d240128c83"}
Jul 10 02:15:16.002530 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:16.002472    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6864ccb968-9dcnh" event={"ID":"9b7250de-fd0e-4f05-9557-9a96d2871b37","Type":"ContainerStarted","Data":"ea51c3464cb8a77b3c86e8c6197b4e581be8f668c9cfc9f6cdfed262960ad97d"}
Jul 10 02:15:16.016957 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:16.016885    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-7fqxc" podStartSLOduration=1.487040837 podStartE2EDuration="3.016870308s" totalImagesPullingTime="1.529829471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:16.016488014 +0000 UTC m=+62.999245703" watchObservedRunningTime="2026-07-10 02:15:16.016870308 +0000 UTC m=+62.999627994"
Jul 10 02:15:16.017919 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:16.017292    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5" podStartSLOduration=3.017276546 podStartE2EDuration="3.017276546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:15.017525152 +0000 UTC m=+62.000282831" watchObservedRunningTime="2026-07-10 02:15:16.017276546 +0000 UTC m=+63.000034234"
Jul 10 02:15:18.011682 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.011628    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6864ccb968-9dcnh" event={"ID":"9b7250de-fd0e-4f05-9557-9a96d2871b37","Type":"ContainerStarted","Data":"6b5b245c23415b40012659f936aa7b46ee8abe16b2f5ccbcf6c308d66e376111"}
Jul 10 02:15:18.031003 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.030938    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6864ccb968-9dcnh" podStartSLOduration=2.828540495 podStartE2EDuration="5.030919112s" totalImagesPullingTime="2.202378617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:18.028944323 +0000 UTC m=+65.011702010" watchObservedRunningTime="2026-07-10 02:15:18.030919112 +0000 UTC m=+65.013676799"
Jul 10 02:15:18.245731 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.245691    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs\") pod \"network-metrics-daemon-dcc8z\" (UID: \"fafe8b6d-e09b-4dac-9687-43f5c7265be7\") " pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:15:18.248275 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.248247    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 02:15:18.258970 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.258943    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fafe8b6d-e09b-4dac-9687-43f5c7265be7-metrics-certs\") pod \"network-metrics-daemon-dcc8z\" (UID: \"fafe8b6d-e09b-4dac-9687-43f5c7265be7\") " pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:15:18.296128 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.296032    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-htmn4\""
Jul 10 02:15:18.301747 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.301719    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dcc8z"
Jul 10 02:15:18.346460 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.346425    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:15:18.346667 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.346547    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6f86\" (UniqueName: \"kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86\") pod \"network-check-target-frlf4\" (UID: \"49e0bcec-546d-488b-8fa5-55a8d7589aca\") " pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:15:18.349585 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.349561    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 02:15:18.349703 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.349599    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 02:15:18.359376 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.359348    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/95280b80-3f30-46c4-be9b-33ff86d5ef24-original-pull-secret\") pod \"global-pull-secret-syncer-w59q2\" (UID: \"95280b80-3f30-46c4-be9b-33ff86d5ef24\") " pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:15:18.359531 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.359493    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 02:15:18.370728 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.370687    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6f86\" (UniqueName: \"kubernetes.io/projected/49e0bcec-546d-488b-8fa5-55a8d7589aca-kube-api-access-q6f86\") pod \"network-check-target-frlf4\" (UID: \"49e0bcec-546d-488b-8fa5-55a8d7589aca\") " pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:15:18.579550 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.579450    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pmctl\""
Jul 10 02:15:18.584861 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.584835    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w59q2"
Jul 10 02:15:18.586465 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.586443    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:15:18.973844 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.973815    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-f7sbz"
Jul 10 02:15:18.989196 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:18.989165    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-frlf4"]
Jul 10 02:15:18.996595 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:15:18.996569    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49e0bcec_546d_488b_8fa5_55a8d7589aca.slice/crio-e92ab39150b192e674372aa964c95b50d1ae4fd07e615e00c5e0239996a27b98 WatchSource:0}: Error finding container e92ab39150b192e674372aa964c95b50d1ae4fd07e615e00c5e0239996a27b98: Status 404 returned error can't find the container with id e92ab39150b192e674372aa964c95b50d1ae4fd07e615e00c5e0239996a27b98
Jul 10 02:15:19.008873 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:19.008851    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w59q2"]
Jul 10 02:15:19.011670 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:19.011642    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dcc8z"]
Jul 10 02:15:19.014551 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:15:19.014478    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95280b80_3f30_46c4_be9b_33ff86d5ef24.slice/crio-73516497703abd51cba3deb6f0031b54bf7d67ed4e10d6e183a4e4a1b7201bb8 WatchSource:0}: Error finding container 73516497703abd51cba3deb6f0031b54bf7d67ed4e10d6e183a4e4a1b7201bb8: Status 404 returned error can't find the container with id 73516497703abd51cba3deb6f0031b54bf7d67ed4e10d6e183a4e4a1b7201bb8
Jul 10 02:15:19.016557 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:15:19.016487    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfafe8b6d_e09b_4dac_9687_43f5c7265be7.slice/crio-f2e1d170bd7d4cb0045bc7d50939904b41e8f590eb2abda1fc24ba04e8290442 WatchSource:0}: Error finding container f2e1d170bd7d4cb0045bc7d50939904b41e8f590eb2abda1fc24ba04e8290442: Status 404 returned error can't find the container with id f2e1d170bd7d4cb0045bc7d50939904b41e8f590eb2abda1fc24ba04e8290442
Jul 10 02:15:19.016960 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:19.016929    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7" event={"ID":"3444d515-1f71-4a24-a1fb-ee8ea3895e28","Type":"ContainerStarted","Data":"107a86eaa8ec81e0ddf970a9fa235b47f8de7c0fb8af3a5b14d04f56bc9bd01b"}
Jul 10 02:15:19.018954 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:19.018927    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-frlf4" event={"ID":"49e0bcec-546d-488b-8fa5-55a8d7589aca","Type":"ContainerStarted","Data":"e92ab39150b192e674372aa964c95b50d1ae4fd07e615e00c5e0239996a27b98"}
Jul 10 02:15:20.027681 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:20.026815    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7" event={"ID":"3444d515-1f71-4a24-a1fb-ee8ea3895e28","Type":"ContainerStarted","Data":"6c2433b01b5b2b727b3d7e5b0666d9b7fcc320b15ea651c6062805281f0410ef"}
Jul 10 02:15:20.027681 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:20.026879    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7" event={"ID":"3444d515-1f71-4a24-a1fb-ee8ea3895e28","Type":"ContainerStarted","Data":"11eb9fcf86ce432f98bbe3f06ec295e2717534e84755d03f18f93db03be3ee07"}
Jul 10 02:15:20.029848 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:20.029822    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w59q2" event={"ID":"95280b80-3f30-46c4-be9b-33ff86d5ef24","Type":"ContainerStarted","Data":"73516497703abd51cba3deb6f0031b54bf7d67ed4e10d6e183a4e4a1b7201bb8"}
Jul 10 02:15:20.031644 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:20.031617    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dcc8z" event={"ID":"fafe8b6d-e09b-4dac-9687-43f5c7265be7","Type":"ContainerStarted","Data":"f2e1d170bd7d4cb0045bc7d50939904b41e8f590eb2abda1fc24ba04e8290442"}
Jul 10 02:15:20.053836 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:20.053783    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4b8477f9-hnpt7" podStartSLOduration=2.629061246 podStartE2EDuration="7.053762797s" totalImagesPullingTime="4.424701551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:20.051847285 +0000 UTC m=+67.034604974" watchObservedRunningTime="2026-07-10 02:15:20.053762797 +0000 UTC m=+67.036520483"
Jul 10 02:15:21.037626 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:21.037586    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dcc8z" event={"ID":"fafe8b6d-e09b-4dac-9687-43f5c7265be7","Type":"ContainerStarted","Data":"e9d816d7420fa1112da7370bf2ed1c2496e342e64c53d51afd7c468cf3444a06"}
Jul 10 02:15:21.038011 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:21.037645    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dcc8z" event={"ID":"fafe8b6d-e09b-4dac-9687-43f5c7265be7","Type":"ContainerStarted","Data":"987b1995dea1d1fa48779dff4a379bc921bd39c6d970e13b7bc266aa48f1f7de"}
Jul 10 02:15:21.075525 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:21.056888    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dcc8z" podStartSLOduration=65.964948033 podStartE2EDuration="1m7.056866803s" totalImagesPullingTime="1.09191877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:14:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:21.055763588 +0000 UTC m=+68.038521275" watchObservedRunningTime="2026-07-10 02:15:21.056866803 +0000 UTC m=+68.039624490"
Jul 10 02:15:22.589159 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:22.589124    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:15:22.591612 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:22.591590    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/de37ac86-4744-475f-bfac-66c48f109148-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vg6lr\" (UID: \"de37ac86-4744-475f-bfac-66c48f109148\") " pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:15:22.855808 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:22.855773    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l6nlg\""
Jul 10 02:15:22.862560 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:22.862535    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vg6lr"
Jul 10 02:15:23.450740 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:23.450707    2579 patch_prober.go:28] interesting pod/image-registry-7b56fb775-gjfpv container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 10 02:15:23.450915 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:23.450760    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7b56fb775-gjfpv" podUID="939f5597-cddf-45a4-80b0-d4392803584f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 02:15:23.978279 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:23.978197    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vg6lr"]
Jul 10 02:15:23.982978 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:15:23.982952    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde37ac86_4744_475f_bfac_66c48f109148.slice/crio-0a5473a57cba5240da67b7143552e3fc6fbccefd86bc69617c85afa4808e71df WatchSource:0}: Error finding container 0a5473a57cba5240da67b7143552e3fc6fbccefd86bc69617c85afa4808e71df: Status 404 returned error can't find the container with id 0a5473a57cba5240da67b7143552e3fc6fbccefd86bc69617c85afa4808e71df
Jul 10 02:15:24.050335 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:24.050297    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vg6lr" event={"ID":"de37ac86-4744-475f-bfac-66c48f109148","Type":"ContainerStarted","Data":"0a5473a57cba5240da67b7143552e3fc6fbccefd86bc69617c85afa4808e71df"}
Jul 10 02:15:24.051848 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:24.051821    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w59q2" event={"ID":"95280b80-3f30-46c4-be9b-33ff86d5ef24","Type":"ContainerStarted","Data":"1351147a8babe074835ed55571c16a12272bea546dd290d29dab003afbb33747"}
Jul 10 02:15:24.053337 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:24.053316    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-frlf4" event={"ID":"49e0bcec-546d-488b-8fa5-55a8d7589aca","Type":"ContainerStarted","Data":"168a84cd5aa808218932dc24d4d0c02414fe1d6ef576cae22784ef0a9006231f"}
Jul 10 02:15:24.053491 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:24.053468    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:15:24.072070 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:24.072008    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-w59q2" podStartSLOduration=65.241423627 podStartE2EDuration="1m10.071989132s" totalImagesPullingTime="4.830565505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:24.069737247 +0000 UTC m=+71.052494932" watchObservedRunningTime="2026-07-10 02:15:24.071989132 +0000 UTC m=+71.054746818"
Jul 10 02:15:24.098243 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:24.098137    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-frlf4" podStartSLOduration=65.25475801 podStartE2EDuration="1m10.098117654s" totalImagesPullingTime="4.843359644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:14:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:24.098034382 +0000 UTC m=+71.080792068" watchObservedRunningTime="2026-07-10 02:15:24.098117654 +0000 UTC m=+71.080875343"
Jul 10 02:15:29.074104 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:29.074072    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vg6lr" event={"ID":"de37ac86-4744-475f-bfac-66c48f109148","Type":"ContainerStarted","Data":"6900115f5a0b21658b81d3f649104308a580fad612dfe046ed6ea485743c55e6"}
Jul 10 02:15:30.079570 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:30.079532    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vg6lr" event={"ID":"de37ac86-4744-475f-bfac-66c48f109148","Type":"ContainerStarted","Data":"69e0567371623b455cc5bd85f6ee2b1c8ec045486c80af57eab59421d386311b"}
Jul 10 02:15:32.092435 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:32.092348    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vg6lr" event={"ID":"de37ac86-4744-475f-bfac-66c48f109148","Type":"ContainerStarted","Data":"46534def194468cbf1a1c108c606d2d57b2ad2c8aaf7901da093d0e8e1b74164"}
Jul 10 02:15:32.112656 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:32.112596    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vg6lr" podStartSLOduration=34.531249264 podStartE2EDuration="42.112579792s" totalImagesPullingTime="7.581330528s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:14:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:32.111730205 +0000 UTC m=+79.094487896" watchObservedRunningTime="2026-07-10 02:15:32.112579792 +0000 UTC m=+79.095337456"
Jul 10 02:15:33.447322 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:33.447297    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:15:34.837892 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.837761    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rbcdz"]
Jul 10 02:15:34.840352 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.840332    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:34.844391 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.844371    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 10 02:15:34.844716 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.844700    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 10 02:15:34.845677 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.845658    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t929n\""
Jul 10 02:15:34.845677 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.845669    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 10 02:15:34.845891 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.845877    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 10 02:15:34.846201 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.846184    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 10 02:15:34.846376 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.846218    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 10 02:15:34.991684 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.991651    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-wtmp\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:34.991684 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.991685    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-textfile\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:34.991936 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.991711    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:34.991936 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.991740    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-accelerators-collector-config\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:34.991936 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.991796    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgqsz\" (UniqueName: \"kubernetes.io/projected/700a1c43-92a0-4541-84fc-5dce3b0a5307-kube-api-access-xgqsz\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:34.991936 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.991828    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/700a1c43-92a0-4541-84fc-5dce3b0a5307-sys\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:34.991936 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.991849    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/700a1c43-92a0-4541-84fc-5dce3b0a5307-root\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:34.991936 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.991869    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/700a1c43-92a0-4541-84fc-5dce3b0a5307-metrics-client-ca\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:34.991936 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:34.991892    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-tls\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.093210 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.093107    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-wtmp\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.093210 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.093159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-textfile\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.093210 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.093183    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.093210 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.093207    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-accelerators-collector-config\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.093564 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.093237    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgqsz\" (UniqueName: \"kubernetes.io/projected/700a1c43-92a0-4541-84fc-5dce3b0a5307-kube-api-access-xgqsz\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.093564 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.093260    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/700a1c43-92a0-4541-84fc-5dce3b0a5307-sys\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.093564 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.093286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/700a1c43-92a0-4541-84fc-5dce3b0a5307-root\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.093564 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.093313    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/700a1c43-92a0-4541-84fc-5dce3b0a5307-metrics-client-ca\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.093564 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.093324    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-wtmp\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.093564 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.093345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-tls\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.093564 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.093375    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/700a1c43-92a0-4541-84fc-5dce3b0a5307-sys\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.093564 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.093390    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/700a1c43-92a0-4541-84fc-5dce3b0a5307-root\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.093564 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:15:35.093449    2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 10 02:15:35.093564 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:15:35.093527    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-tls podName:700a1c43-92a0-4541-84fc-5dce3b0a5307 nodeName:}" failed. No retries permitted until 2026-07-10 02:15:35.593487941 +0000 UTC m=+82.576245619 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-tls") pod "node-exporter-rbcdz" (UID: "700a1c43-92a0-4541-84fc-5dce3b0a5307") : secret "node-exporter-tls" not found
Jul 10 02:15:35.094043 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.093942    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-accelerators-collector-config\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.094115 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.094096    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-textfile\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.094318 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.094301    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/700a1c43-92a0-4541-84fc-5dce3b0a5307-metrics-client-ca\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.096144 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.096115    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.112428 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.112403    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgqsz\" (UniqueName: \"kubernetes.io/projected/700a1c43-92a0-4541-84fc-5dce3b0a5307-kube-api-access-xgqsz\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.599609 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.599571    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-tls\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.602088 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.602068    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/700a1c43-92a0-4541-84fc-5dce3b0a5307-node-exporter-tls\") pod \"node-exporter-rbcdz\" (UID: \"700a1c43-92a0-4541-84fc-5dce3b0a5307\") " pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.751121 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:35.751087    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rbcdz"
Jul 10 02:15:35.761630 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:15:35.761605    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod700a1c43_92a0_4541_84fc_5dce3b0a5307.slice/crio-5c8b3eb58f87593cbfe8b10452a0e811069b2b8945b1af132482bc42e3071c90 WatchSource:0}: Error finding container 5c8b3eb58f87593cbfe8b10452a0e811069b2b8945b1af132482bc42e3071c90: Status 404 returned error can't find the container with id 5c8b3eb58f87593cbfe8b10452a0e811069b2b8945b1af132482bc42e3071c90
Jul 10 02:15:36.007141 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:36.007116    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f7d7fccf8-ps2l5"
Jul 10 02:15:36.106906 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:36.106869    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rbcdz" event={"ID":"700a1c43-92a0-4541-84fc-5dce3b0a5307","Type":"ContainerStarted","Data":"5c8b3eb58f87593cbfe8b10452a0e811069b2b8945b1af132482bc42e3071c90"}
Jul 10 02:15:37.111951 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:37.111916    2579 generic.go:356] "Generic (PLEG): container finished" podID="700a1c43-92a0-4541-84fc-5dce3b0a5307" containerID="86cced776c66c7ba11e1f8ce0125faf64fad266c9b6dd7b757152645ecc08264" exitCode=0
Jul 10 02:15:37.112376 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:37.111998    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rbcdz" event={"ID":"700a1c43-92a0-4541-84fc-5dce3b0a5307","Type":"ContainerDied","Data":"86cced776c66c7ba11e1f8ce0125faf64fad266c9b6dd7b757152645ecc08264"}
Jul 10 02:15:38.118605 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.118560    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rbcdz" event={"ID":"700a1c43-92a0-4541-84fc-5dce3b0a5307","Type":"ContainerStarted","Data":"2e924a587513321fcacb98ab25437fc0dec1bb9ca977834a9db021848023716c"}
Jul 10 02:15:38.119048 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.118629    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rbcdz" event={"ID":"700a1c43-92a0-4541-84fc-5dce3b0a5307","Type":"ContainerStarted","Data":"ab0a06acd6cc523af9ab24675c29658fdd3ad0614f48aa9f0d8558f498089cb8"}
Jul 10 02:15:38.140837 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.140793    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rbcdz" podStartSLOduration=3.052854648 podStartE2EDuration="4.140776455s" totalImagesPullingTime="1.017319368s" totalInitContainerRuntime="70.602439ms" isStatefulPod=true podCreationTimestamp="2026-07-10 02:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:38.139059967 +0000 UTC m=+85.121817676" watchObservedRunningTime="2026-07-10 02:15:38.140776455 +0000 UTC m=+85.123534155"
Jul 10 02:15:38.463011 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.462912    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7b56fb775-gjfpv" podUID="939f5597-cddf-45a4-80b0-d4392803584f" containerName="registry" containerID="cri-o://8fe7c07d2417748a04fd744f6f116ef8479fa789fad1e9cdf58b31754a18faf4" gracePeriod=30
Jul 10 02:15:38.701473 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.701449    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:15:38.825237 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.825155    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgtz2\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-kube-api-access-dgtz2\") pod \"939f5597-cddf-45a4-80b0-d4392803584f\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") "
Jul 10 02:15:38.825237 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.825190    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/939f5597-cddf-45a4-80b0-d4392803584f-ca-trust-extracted\") pod \"939f5597-cddf-45a4-80b0-d4392803584f\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") "
Jul 10 02:15:38.825237 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.825213    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/939f5597-cddf-45a4-80b0-d4392803584f-trusted-ca\") pod \"939f5597-cddf-45a4-80b0-d4392803584f\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") "
Jul 10 02:15:38.825530 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.825256    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/939f5597-cddf-45a4-80b0-d4392803584f-registry-certificates\") pod \"939f5597-cddf-45a4-80b0-d4392803584f\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") "
Jul 10 02:15:38.825530 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.825281    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls\") pod \"939f5597-cddf-45a4-80b0-d4392803584f\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") "
Jul 10 02:15:38.825530 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.825309    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/939f5597-cddf-45a4-80b0-d4392803584f-installation-pull-secrets\") pod \"939f5597-cddf-45a4-80b0-d4392803584f\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") "
Jul 10 02:15:38.825530 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.825404    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-bound-sa-token\") pod \"939f5597-cddf-45a4-80b0-d4392803584f\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") "
Jul 10 02:15:38.825530 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.825467    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/939f5597-cddf-45a4-80b0-d4392803584f-image-registry-private-configuration\") pod \"939f5597-cddf-45a4-80b0-d4392803584f\" (UID: \"939f5597-cddf-45a4-80b0-d4392803584f\") "
Jul 10 02:15:38.825806 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.825779    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/939f5597-cddf-45a4-80b0-d4392803584f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "939f5597-cddf-45a4-80b0-d4392803584f" (UID: "939f5597-cddf-45a4-80b0-d4392803584f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:15:38.825806 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.825788    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/939f5597-cddf-45a4-80b0-d4392803584f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "939f5597-cddf-45a4-80b0-d4392803584f" (UID: "939f5597-cddf-45a4-80b0-d4392803584f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:15:38.827976 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.827954    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-kube-api-access-dgtz2" (OuterVolumeSpecName: "kube-api-access-dgtz2") pod "939f5597-cddf-45a4-80b0-d4392803584f" (UID: "939f5597-cddf-45a4-80b0-d4392803584f"). InnerVolumeSpecName "kube-api-access-dgtz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:15:38.828111 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.828042    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "939f5597-cddf-45a4-80b0-d4392803584f" (UID: "939f5597-cddf-45a4-80b0-d4392803584f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:15:38.828267 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.828243    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "939f5597-cddf-45a4-80b0-d4392803584f" (UID: "939f5597-cddf-45a4-80b0-d4392803584f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:15:38.828361 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.828306    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/939f5597-cddf-45a4-80b0-d4392803584f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "939f5597-cddf-45a4-80b0-d4392803584f" (UID: "939f5597-cddf-45a4-80b0-d4392803584f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:15:38.828398 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.828379    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/939f5597-cddf-45a4-80b0-d4392803584f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "939f5597-cddf-45a4-80b0-d4392803584f" (UID: "939f5597-cddf-45a4-80b0-d4392803584f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:15:38.834095 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.834075    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/939f5597-cddf-45a4-80b0-d4392803584f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "939f5597-cddf-45a4-80b0-d4392803584f" (UID: "939f5597-cddf-45a4-80b0-d4392803584f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:15:38.927071 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.927036    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/939f5597-cddf-45a4-80b0-d4392803584f-registry-certificates\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:15:38.927071 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.927064    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-registry-tls\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:15:38.927071 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.927077    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/939f5597-cddf-45a4-80b0-d4392803584f-installation-pull-secrets\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:15:38.927071 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.927085    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-bound-sa-token\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:15:38.927318 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.927094    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/939f5597-cddf-45a4-80b0-d4392803584f-image-registry-private-configuration\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:15:38.927318 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.927105    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgtz2\" (UniqueName: \"kubernetes.io/projected/939f5597-cddf-45a4-80b0-d4392803584f-kube-api-access-dgtz2\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:15:38.927318 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.927112    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/939f5597-cddf-45a4-80b0-d4392803584f-ca-trust-extracted\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:15:38.927318 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:38.927120    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/939f5597-cddf-45a4-80b0-d4392803584f-trusted-ca\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:15:39.123071 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:39.123041    2579 generic.go:356] "Generic (PLEG): container finished" podID="939f5597-cddf-45a4-80b0-d4392803584f" containerID="8fe7c07d2417748a04fd744f6f116ef8479fa789fad1e9cdf58b31754a18faf4" exitCode=0
Jul 10 02:15:39.123548 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:39.123094    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b56fb775-gjfpv"
Jul 10 02:15:39.123548 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:39.123141    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b56fb775-gjfpv" event={"ID":"939f5597-cddf-45a4-80b0-d4392803584f","Type":"ContainerDied","Data":"8fe7c07d2417748a04fd744f6f116ef8479fa789fad1e9cdf58b31754a18faf4"}
Jul 10 02:15:39.123548 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:39.123202    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b56fb775-gjfpv" event={"ID":"939f5597-cddf-45a4-80b0-d4392803584f","Type":"ContainerDied","Data":"fad18897d3853bf92b92593c3110349b069573855102e75f597d07854800ec07"}
Jul 10 02:15:39.123548 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:39.123236    2579 scope.go:117] "RemoveContainer" containerID="8fe7c07d2417748a04fd744f6f116ef8479fa789fad1e9cdf58b31754a18faf4"
Jul 10 02:15:39.135843 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:39.135824    2579 scope.go:117] "RemoveContainer" containerID="8fe7c07d2417748a04fd744f6f116ef8479fa789fad1e9cdf58b31754a18faf4"
Jul 10 02:15:39.136150 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:15:39.136128    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fe7c07d2417748a04fd744f6f116ef8479fa789fad1e9cdf58b31754a18faf4\": container with ID starting with 8fe7c07d2417748a04fd744f6f116ef8479fa789fad1e9cdf58b31754a18faf4 not found: ID does not exist" containerID="8fe7c07d2417748a04fd744f6f116ef8479fa789fad1e9cdf58b31754a18faf4"
Jul 10 02:15:39.136190 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:39.136159    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fe7c07d2417748a04fd744f6f116ef8479fa789fad1e9cdf58b31754a18faf4"} err="failed to get container status \"8fe7c07d2417748a04fd744f6f116ef8479fa789fad1e9cdf58b31754a18faf4\": rpc error: code = NotFound desc = could not find container \"8fe7c07d2417748a04fd744f6f116ef8479fa789fad1e9cdf58b31754a18faf4\": container with ID starting with 8fe7c07d2417748a04fd744f6f116ef8479fa789fad1e9cdf58b31754a18faf4 not found: ID does not exist"
Jul 10 02:15:39.146474 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:39.146453    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b56fb775-gjfpv"]
Jul 10 02:15:39.152053 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:39.152035    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7b56fb775-gjfpv"]
Jul 10 02:15:39.565429 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:39.565345    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="939f5597-cddf-45a4-80b0-d4392803584f" path="/var/lib/kubelet/pods/939f5597-cddf-45a4-80b0-d4392803584f/volumes"
Jul 10 02:15:55.059893 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:15:55.059863    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-frlf4"
Jul 10 02:16:00.578089 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:16:00.578063    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rbcdz_700a1c43-92a0-4541-84fc-5dce3b0a5307/init-textfile/0.log"
Jul 10 02:16:00.779223 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:16:00.779192    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rbcdz_700a1c43-92a0-4541-84fc-5dce3b0a5307/node-exporter/0.log"
Jul 10 02:16:00.978751 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:16:00.978725    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rbcdz_700a1c43-92a0-4541-84fc-5dce3b0a5307/kube-rbac-proxy/0.log"
Jul 10 02:16:05.578473 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:16:05.578440    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-7fqxc_a07b77b4-3144-4cbb-8937-0a6bd23da2c0/networking-console-plugin/0.log"
Jul 10 02:19:01.309833 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.309803    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"]
Jul 10 02:19:01.310267 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.310052    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="939f5597-cddf-45a4-80b0-d4392803584f" containerName="registry"
Jul 10 02:19:01.310267 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.310062    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="939f5597-cddf-45a4-80b0-d4392803584f" containerName="registry"
Jul 10 02:19:01.310267 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.310103    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="939f5597-cddf-45a4-80b0-d4392803584f" containerName="registry"
Jul 10 02:19:01.312057 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.312040    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.314992 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.314968    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 10 02:19:01.315102 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.315090    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 10 02:19:01.315284 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.315166    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 10 02:19:01.315550 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.315526    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-p8792\""
Jul 10 02:19:01.315550 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.315529    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 10 02:19:01.324045 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.324022    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"]
Jul 10 02:19:01.467251 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.467222    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b75e848a-4e89-46f3-bdb4-7224a110f1c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-jszsk\" (UID: \"b75e848a-4e89-46f3-bdb4-7224a110f1c1\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.467409 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.467258    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b75e848a-4e89-46f3-bdb4-7224a110f1c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-jszsk\" (UID: \"b75e848a-4e89-46f3-bdb4-7224a110f1c1\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.467409 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.467283    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wtsc\" (UniqueName: \"kubernetes.io/projected/b75e848a-4e89-46f3-bdb4-7224a110f1c1-kube-api-access-9wtsc\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-jszsk\" (UID: \"b75e848a-4e89-46f3-bdb4-7224a110f1c1\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.467409 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.467369    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b75e848a-4e89-46f3-bdb4-7224a110f1c1-manifests\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-jszsk\" (UID: \"b75e848a-4e89-46f3-bdb4-7224a110f1c1\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.467530 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.467410    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b75e848a-4e89-46f3-bdb4-7224a110f1c1-tmp\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-jszsk\" (UID: \"b75e848a-4e89-46f3-bdb4-7224a110f1c1\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.568294 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.568221    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b75e848a-4e89-46f3-bdb4-7224a110f1c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-jszsk\" (UID: \"b75e848a-4e89-46f3-bdb4-7224a110f1c1\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.568294 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.568257    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wtsc\" (UniqueName: \"kubernetes.io/projected/b75e848a-4e89-46f3-bdb4-7224a110f1c1-kube-api-access-9wtsc\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-jszsk\" (UID: \"b75e848a-4e89-46f3-bdb4-7224a110f1c1\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.568454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.568437    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b75e848a-4e89-46f3-bdb4-7224a110f1c1-manifests\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-jszsk\" (UID: \"b75e848a-4e89-46f3-bdb4-7224a110f1c1\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.568490 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.568482    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b75e848a-4e89-46f3-bdb4-7224a110f1c1-tmp\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-jszsk\" (UID: \"b75e848a-4e89-46f3-bdb4-7224a110f1c1\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.568559 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.568547    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b75e848a-4e89-46f3-bdb4-7224a110f1c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-jszsk\" (UID: \"b75e848a-4e89-46f3-bdb4-7224a110f1c1\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.568815 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.568789    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b75e848a-4e89-46f3-bdb4-7224a110f1c1-manifests\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-jszsk\" (UID: \"b75e848a-4e89-46f3-bdb4-7224a110f1c1\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.570783 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.570757    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b75e848a-4e89-46f3-bdb4-7224a110f1c1-tmp\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-jszsk\" (UID: \"b75e848a-4e89-46f3-bdb4-7224a110f1c1\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.570876 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.570827    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b75e848a-4e89-46f3-bdb4-7224a110f1c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-jszsk\" (UID: \"b75e848a-4e89-46f3-bdb4-7224a110f1c1\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.570914 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.570899    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b75e848a-4e89-46f3-bdb4-7224a110f1c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-jszsk\" (UID: \"b75e848a-4e89-46f3-bdb4-7224a110f1c1\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.579636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.579613    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wtsc\" (UniqueName: \"kubernetes.io/projected/b75e848a-4e89-46f3-bdb4-7224a110f1c1-kube-api-access-9wtsc\") pod \"opendatahub-operator-controller-manager-94f6fbcc5-jszsk\" (UID: \"b75e848a-4e89-46f3-bdb4-7224a110f1c1\") " pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.622215 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.622187    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:01.756054 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.756026    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"]
Jul 10 02:19:01.759092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:19:01.759063    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb75e848a_4e89_46f3_bdb4_7224a110f1c1.slice/crio-499147d3c559a13c75c4b05a44c47503f4553074233921280f44308c321a30c5 WatchSource:0}: Error finding container 499147d3c559a13c75c4b05a44c47503f4553074233921280f44308c321a30c5: Status 404 returned error can't find the container with id 499147d3c559a13c75c4b05a44c47503f4553074233921280f44308c321a30c5
Jul 10 02:19:01.799855 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:01.799820    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk" event={"ID":"b75e848a-4e89-46f3-bdb4-7224a110f1c1","Type":"ContainerStarted","Data":"499147d3c559a13c75c4b05a44c47503f4553074233921280f44308c321a30c5"}
Jul 10 02:19:05.817628 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:05.817587    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk" event={"ID":"b75e848a-4e89-46f3-bdb4-7224a110f1c1","Type":"ContainerStarted","Data":"1937c2f966e8eb06c2031c391ccca22ff99f5fd787b51fea5e686b8d7427d40a"}
Jul 10 02:19:06.822846 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:06.822810    2579 generic.go:356] "Generic (PLEG): container finished" podID="b75e848a-4e89-46f3-bdb4-7224a110f1c1" containerID="1937c2f966e8eb06c2031c391ccca22ff99f5fd787b51fea5e686b8d7427d40a" exitCode=0
Jul 10 02:19:06.823215 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:06.822886    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk" event={"ID":"b75e848a-4e89-46f3-bdb4-7224a110f1c1","Type":"ContainerDied","Data":"1937c2f966e8eb06c2031c391ccca22ff99f5fd787b51fea5e686b8d7427d40a"}
Jul 10 02:19:07.828360 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:07.828327    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk" event={"ID":"b75e848a-4e89-46f3-bdb4-7224a110f1c1","Type":"ContainerStarted","Data":"d2b85eb603aa8b38a592634fc8a00732d2c6248226e9b4bb0862a3e744a95d9e"}
Jul 10 02:19:07.828750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:07.828464    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:07.851752 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:07.851709    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk" podStartSLOduration=2.321484874 podStartE2EDuration="6.851695977s" totalImagesPullingTime="3.533732609s" totalInitContainerRuntime="996.478494ms" isStatefulPod=false podCreationTimestamp="2026-07-10 02:19:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:19:07.850467367 +0000 UTC m=+294.833225077" watchObservedRunningTime="2026-07-10 02:19:07.851695977 +0000 UTC m=+294.834453663"
Jul 10 02:19:13.428567 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:13.428538    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 10 02:19:18.834067 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:18.834034    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-94f6fbcc5-jszsk"
Jul 10 02:19:21.092268 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.092235    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-779d665c94-qbtkd"]
Jul 10 02:19:21.094399 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.094383    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-779d665c94-qbtkd"
Jul 10 02:19:21.097316 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.097286    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 10 02:19:21.097316 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.097301    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 10 02:19:21.097582 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.097562    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 10 02:19:21.097668 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.097655    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 10 02:19:21.098684 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.098670    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wwzv9\""
Jul 10 02:19:21.113608 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.113585    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-779d665c94-qbtkd"]
Jul 10 02:19:21.115612 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.115591    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6qvh\" (UniqueName: \"kubernetes.io/projected/9602c94d-88e4-47f4-af44-562d46baa3b8-kube-api-access-d6qvh\") pod \"kube-auth-proxy-779d665c94-qbtkd\" (UID: \"9602c94d-88e4-47f4-af44-562d46baa3b8\") " pod="openshift-ingress/kube-auth-proxy-779d665c94-qbtkd"
Jul 10 02:19:21.115715 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.115622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9602c94d-88e4-47f4-af44-562d46baa3b8-tls-certs\") pod \"kube-auth-proxy-779d665c94-qbtkd\" (UID: \"9602c94d-88e4-47f4-af44-562d46baa3b8\") " pod="openshift-ingress/kube-auth-proxy-779d665c94-qbtkd"
Jul 10 02:19:21.115715 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.115695    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9602c94d-88e4-47f4-af44-562d46baa3b8-tmp\") pod \"kube-auth-proxy-779d665c94-qbtkd\" (UID: \"9602c94d-88e4-47f4-af44-562d46baa3b8\") " pod="openshift-ingress/kube-auth-proxy-779d665c94-qbtkd"
Jul 10 02:19:21.216561 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.216531    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6qvh\" (UniqueName: \"kubernetes.io/projected/9602c94d-88e4-47f4-af44-562d46baa3b8-kube-api-access-d6qvh\") pod \"kube-auth-proxy-779d665c94-qbtkd\" (UID: \"9602c94d-88e4-47f4-af44-562d46baa3b8\") " pod="openshift-ingress/kube-auth-proxy-779d665c94-qbtkd"
Jul 10 02:19:21.216728 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.216568    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9602c94d-88e4-47f4-af44-562d46baa3b8-tls-certs\") pod \"kube-auth-proxy-779d665c94-qbtkd\" (UID: \"9602c94d-88e4-47f4-af44-562d46baa3b8\") " pod="openshift-ingress/kube-auth-proxy-779d665c94-qbtkd"
Jul 10 02:19:21.216728 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.216618    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9602c94d-88e4-47f4-af44-562d46baa3b8-tmp\") pod \"kube-auth-proxy-779d665c94-qbtkd\" (UID: \"9602c94d-88e4-47f4-af44-562d46baa3b8\") " pod="openshift-ingress/kube-auth-proxy-779d665c94-qbtkd"
Jul 10 02:19:21.218992 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.218970    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9602c94d-88e4-47f4-af44-562d46baa3b8-tmp\") pod \"kube-auth-proxy-779d665c94-qbtkd\" (UID: \"9602c94d-88e4-47f4-af44-562d46baa3b8\") " pod="openshift-ingress/kube-auth-proxy-779d665c94-qbtkd"
Jul 10 02:19:21.219126 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.219107    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9602c94d-88e4-47f4-af44-562d46baa3b8-tls-certs\") pod \"kube-auth-proxy-779d665c94-qbtkd\" (UID: \"9602c94d-88e4-47f4-af44-562d46baa3b8\") " pod="openshift-ingress/kube-auth-proxy-779d665c94-qbtkd"
Jul 10 02:19:21.228560 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.228528    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-865944fd75-qj999"]
Jul 10 02:19:21.231309 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.231290    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:21.232432 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.232407    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6qvh\" (UniqueName: \"kubernetes.io/projected/9602c94d-88e4-47f4-af44-562d46baa3b8-kube-api-access-d6qvh\") pod \"kube-auth-proxy-779d665c94-qbtkd\" (UID: \"9602c94d-88e4-47f4-af44-562d46baa3b8\") " pod="openshift-ingress/kube-auth-proxy-779d665c94-qbtkd"
Jul 10 02:19:21.233954 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.233932    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 10 02:19:21.234311 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.234285    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 10 02:19:21.234311 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.234304    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kdq8g\""
Jul 10 02:19:21.234471 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.234303    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 10 02:19:21.234732 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.234710    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 10 02:19:21.235055 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.235041    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 10 02:19:21.242975 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.242953    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-865944fd75-qj999"]
Jul 10 02:19:21.316963 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.316920    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7e66e036-1b0f-4b5d-9555-4b5f356ef186-metrics-cert\") pod \"lws-controller-manager-865944fd75-qj999\" (UID: \"7e66e036-1b0f-4b5d-9555-4b5f356ef186\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:21.316963 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.316956    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z4n2\" (UniqueName: \"kubernetes.io/projected/7e66e036-1b0f-4b5d-9555-4b5f356ef186-kube-api-access-6z4n2\") pod \"lws-controller-manager-865944fd75-qj999\" (UID: \"7e66e036-1b0f-4b5d-9555-4b5f356ef186\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:21.317219 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.317029    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7e66e036-1b0f-4b5d-9555-4b5f356ef186-manager-config\") pod \"lws-controller-manager-865944fd75-qj999\" (UID: \"7e66e036-1b0f-4b5d-9555-4b5f356ef186\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:21.317219 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.317065    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e66e036-1b0f-4b5d-9555-4b5f356ef186-cert\") pod \"lws-controller-manager-865944fd75-qj999\" (UID: \"7e66e036-1b0f-4b5d-9555-4b5f356ef186\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:21.404063 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.404024    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-779d665c94-qbtkd"
Jul 10 02:19:21.418267 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.418235    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7e66e036-1b0f-4b5d-9555-4b5f356ef186-manager-config\") pod \"lws-controller-manager-865944fd75-qj999\" (UID: \"7e66e036-1b0f-4b5d-9555-4b5f356ef186\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:21.418413 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.418284    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e66e036-1b0f-4b5d-9555-4b5f356ef186-cert\") pod \"lws-controller-manager-865944fd75-qj999\" (UID: \"7e66e036-1b0f-4b5d-9555-4b5f356ef186\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:21.418413 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.418351    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7e66e036-1b0f-4b5d-9555-4b5f356ef186-metrics-cert\") pod \"lws-controller-manager-865944fd75-qj999\" (UID: \"7e66e036-1b0f-4b5d-9555-4b5f356ef186\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:21.418413 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.418376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z4n2\" (UniqueName: \"kubernetes.io/projected/7e66e036-1b0f-4b5d-9555-4b5f356ef186-kube-api-access-6z4n2\") pod \"lws-controller-manager-865944fd75-qj999\" (UID: \"7e66e036-1b0f-4b5d-9555-4b5f356ef186\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:21.419077 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.419041    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7e66e036-1b0f-4b5d-9555-4b5f356ef186-manager-config\") pod \"lws-controller-manager-865944fd75-qj999\" (UID: \"7e66e036-1b0f-4b5d-9555-4b5f356ef186\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:21.420964 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.420940    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e66e036-1b0f-4b5d-9555-4b5f356ef186-cert\") pod \"lws-controller-manager-865944fd75-qj999\" (UID: \"7e66e036-1b0f-4b5d-9555-4b5f356ef186\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:21.421170 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.421147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7e66e036-1b0f-4b5d-9555-4b5f356ef186-metrics-cert\") pod \"lws-controller-manager-865944fd75-qj999\" (UID: \"7e66e036-1b0f-4b5d-9555-4b5f356ef186\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:21.441425 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.441376    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z4n2\" (UniqueName: \"kubernetes.io/projected/7e66e036-1b0f-4b5d-9555-4b5f356ef186-kube-api-access-6z4n2\") pod \"lws-controller-manager-865944fd75-qj999\" (UID: \"7e66e036-1b0f-4b5d-9555-4b5f356ef186\") " pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:21.535219 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.535196    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-779d665c94-qbtkd"]
Jul 10 02:19:21.537470 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:19:21.537439    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9602c94d_88e4_47f4_af44_562d46baa3b8.slice/crio-582a65ea276396e11e78ec2b447b459b61997478b9386478e1683953d649dbf7 WatchSource:0}: Error finding container 582a65ea276396e11e78ec2b447b459b61997478b9386478e1683953d649dbf7: Status 404 returned error can't find the container with id 582a65ea276396e11e78ec2b447b459b61997478b9386478e1683953d649dbf7
Jul 10 02:19:21.539262 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.539249    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:19:21.548756 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.548734    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:21.681353 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.681330    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-865944fd75-qj999"]
Jul 10 02:19:21.683172 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:19:21.683146    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e66e036_1b0f_4b5d_9555_4b5f356ef186.slice/crio-66b76d37648d98c9527ea4c9cbf04b0db3668ff48dda57276f3e0ad39f88b4f6 WatchSource:0}: Error finding container 66b76d37648d98c9527ea4c9cbf04b0db3668ff48dda57276f3e0ad39f88b4f6: Status 404 returned error can't find the container with id 66b76d37648d98c9527ea4c9cbf04b0db3668ff48dda57276f3e0ad39f88b4f6
Jul 10 02:19:21.877714 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.877674    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999" event={"ID":"7e66e036-1b0f-4b5d-9555-4b5f356ef186","Type":"ContainerStarted","Data":"66b76d37648d98c9527ea4c9cbf04b0db3668ff48dda57276f3e0ad39f88b4f6"}
Jul 10 02:19:21.878912 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:21.878888    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-779d665c94-qbtkd" event={"ID":"9602c94d-88e4-47f4-af44-562d46baa3b8","Type":"ContainerStarted","Data":"582a65ea276396e11e78ec2b447b459b61997478b9386478e1683953d649dbf7"}
Jul 10 02:19:25.899971 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:25.899936    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999" event={"ID":"7e66e036-1b0f-4b5d-9555-4b5f356ef186","Type":"ContainerStarted","Data":"9e701eb9e77449b02837a3b7704b99bef8553b8882b02f34fe549a8a29534b39"}
Jul 10 02:19:25.900490 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:25.900021    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:25.901565 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:25.901542    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-779d665c94-qbtkd" event={"ID":"9602c94d-88e4-47f4-af44-562d46baa3b8","Type":"ContainerStarted","Data":"2d18ccca9e658220dca022c7cbf6172291b768a0b046bfb8578708037221288c"}
Jul 10 02:19:25.918135 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:25.918085    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999" podStartSLOduration=1.6586738109999999 podStartE2EDuration="4.918071382s" totalImagesPullingTime="3.259397571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:19:25.916828216 +0000 UTC m=+312.899585914" watchObservedRunningTime="2026-07-10 02:19:25.918071382 +0000 UTC m=+312.900829068"
Jul 10 02:19:36.906949 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:36.906917    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-865944fd75-qj999"
Jul 10 02:19:36.969308 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:19:36.969249    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-779d665c94-qbtkd" podStartSLOduration=12.561719404 podStartE2EDuration="15.969232958s" totalImagesPullingTime="3.407513554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:19:25.931747686 +0000 UTC m=+312.914505372" watchObservedRunningTime="2026-07-10 02:19:36.969232958 +0000 UTC m=+323.951990643"
Jul 10 02:21:21.201646 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.201609    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"]
Jul 10 02:21:21.203763 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.203747    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:21.206458 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.206435    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 02:21:21.206642 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.206465    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f5mnk\""
Jul 10 02:21:21.206642 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.206468    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 02:21:21.226449 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.226426    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"]
Jul 10 02:21:21.326638 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.326610    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljdzb\" (UniqueName: \"kubernetes.io/projected/d2a3cd42-0809-4cef-8d3a-398e078a0e6f-kube-api-access-ljdzb\") pod \"kuadrant-operator-controller-manager-55467d9f7-b998b\" (UID: \"d2a3cd42-0809-4cef-8d3a-398e078a0e6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:21.326794 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.326653    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2a3cd42-0809-4cef-8d3a-398e078a0e6f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-b998b\" (UID: \"d2a3cd42-0809-4cef-8d3a-398e078a0e6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:21.428044 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.427989    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljdzb\" (UniqueName: \"kubernetes.io/projected/d2a3cd42-0809-4cef-8d3a-398e078a0e6f-kube-api-access-ljdzb\") pod \"kuadrant-operator-controller-manager-55467d9f7-b998b\" (UID: \"d2a3cd42-0809-4cef-8d3a-398e078a0e6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:21.428232 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.428061    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2a3cd42-0809-4cef-8d3a-398e078a0e6f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-b998b\" (UID: \"d2a3cd42-0809-4cef-8d3a-398e078a0e6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:21.428435 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.428415    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2a3cd42-0809-4cef-8d3a-398e078a0e6f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-b998b\" (UID: \"d2a3cd42-0809-4cef-8d3a-398e078a0e6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:21.436659 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.436631    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljdzb\" (UniqueName: \"kubernetes.io/projected/d2a3cd42-0809-4cef-8d3a-398e078a0e6f-kube-api-access-ljdzb\") pod \"kuadrant-operator-controller-manager-55467d9f7-b998b\" (UID: \"d2a3cd42-0809-4cef-8d3a-398e078a0e6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:21.513606 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.513529    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:21.577647 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.577615    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"]
Jul 10 02:21:21.581369 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.581308    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"]
Jul 10 02:21:21.599800 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.599777    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"]
Jul 10 02:21:21.602177 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.602160    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"
Jul 10 02:21:21.614932 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.614908    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"]
Jul 10 02:21:21.730750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.730707    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6k2b\" (UniqueName: \"kubernetes.io/projected/9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962-kube-api-access-v6k2b\") pod \"kuadrant-operator-controller-manager-55467d9f7-pkj2v\" (UID: \"9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"
Jul 10 02:21:21.730750 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.730759    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-pkj2v\" (UID: \"9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"
Jul 10 02:21:21.763127 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:21:21.763094    2579 log.go:32] "RunPodSandbox from runtime service failed" err=<
Jul 10 02:21:21.763127 ip-10-0-132-62 kubenswrapper[2579]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-b998b_kuadrant-system_d2a3cd42-0809-4cef-8d3a-398e078a0e6f_0(af9a2c28c47c1bffd21cb034192e07b1099d448e6d84d3320429066e96fdcc8d): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-b998b to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"af9a2c28c47c1bffd21cb034192e07b1099d448e6d84d3320429066e96fdcc8d" Netns:"/var/run/netns/a8dbf601-235c-4780-8df8-27831656ff36" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-b998b;K8S_POD_INFRA_CONTAINER_ID=af9a2c28c47c1bffd21cb034192e07b1099d448e6d84d3320429066e96fdcc8d;K8S_POD_UID=d2a3cd42-0809-4cef-8d3a-398e078a0e6f" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b/d2a3cd42-0809-4cef-8d3a-398e078a0e6f:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b af9a2c28c47c1bffd21cb034192e07b1099d448e6d84d3320429066e96fdcc8d network default NAD default NAD key default] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b af9a2c28c47c1bffd21cb034192e07b1099d448e6d84d3320429066e96fdcc8d network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:85:00:13 [10.133.0.19/23]
Jul 10 02:21:21.763127 ip-10-0-132-62 kubenswrapper[2579]:         '
Jul 10 02:21:21.763127 ip-10-0-132-62 kubenswrapper[2579]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Jul 10 02:21:21.763127 ip-10-0-132-62 kubenswrapper[2579]:  >
Jul 10 02:21:21.763371 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:21:21.763154    2579 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=<
Jul 10 02:21:21.763371 ip-10-0-132-62 kubenswrapper[2579]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-b998b_kuadrant-system_d2a3cd42-0809-4cef-8d3a-398e078a0e6f_0(af9a2c28c47c1bffd21cb034192e07b1099d448e6d84d3320429066e96fdcc8d): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-b998b to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"af9a2c28c47c1bffd21cb034192e07b1099d448e6d84d3320429066e96fdcc8d" Netns:"/var/run/netns/a8dbf601-235c-4780-8df8-27831656ff36" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-b998b;K8S_POD_INFRA_CONTAINER_ID=af9a2c28c47c1bffd21cb034192e07b1099d448e6d84d3320429066e96fdcc8d;K8S_POD_UID=d2a3cd42-0809-4cef-8d3a-398e078a0e6f" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b/d2a3cd42-0809-4cef-8d3a-398e078a0e6f:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b af9a2c28c47c1bffd21cb034192e07b1099d448e6d84d3320429066e96fdcc8d network default NAD default NAD key default] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b af9a2c28c47c1bffd21cb034192e07b1099d448e6d84d3320429066e96fdcc8d network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:85:00:13 [10.133.0.19/23]
Jul 10 02:21:21.763371 ip-10-0-132-62 kubenswrapper[2579]:         '
Jul 10 02:21:21.763371 ip-10-0-132-62 kubenswrapper[2579]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Jul 10 02:21:21.763371 ip-10-0-132-62 kubenswrapper[2579]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:21.831621 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.831583    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6k2b\" (UniqueName: \"kubernetes.io/projected/9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962-kube-api-access-v6k2b\") pod \"kuadrant-operator-controller-manager-55467d9f7-pkj2v\" (UID: \"9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"
Jul 10 02:21:21.831621 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.831634    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-pkj2v\" (UID: \"9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"
Jul 10 02:21:21.832033 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.832015    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-pkj2v\" (UID: \"9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"
Jul 10 02:21:21.843129 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.843112    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6k2b\" (UniqueName: \"kubernetes.io/projected/9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962-kube-api-access-v6k2b\") pod \"kuadrant-operator-controller-manager-55467d9f7-pkj2v\" (UID: \"9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"
Jul 10 02:21:21.913069 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:21.913043    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"
Jul 10 02:21:22.048055 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:22.048034    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"]
Jul 10 02:21:22.049726 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:21:22.049699    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d5f4b3a_497b_4e3a_8acb_0cf7f2e05962.slice/crio-8f0b1b4e254a60b174367cca0d29735edee4157dcd6f500900c2cb0faa4ac631 WatchSource:0}: Error finding container 8f0b1b4e254a60b174367cca0d29735edee4157dcd6f500900c2cb0faa4ac631: Status 404 returned error can't find the container with id 8f0b1b4e254a60b174367cca0d29735edee4157dcd6f500900c2cb0faa4ac631
Jul 10 02:21:22.309291 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:22.309199    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v" event={"ID":"9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962","Type":"ContainerStarted","Data":"8f0b1b4e254a60b174367cca0d29735edee4157dcd6f500900c2cb0faa4ac631"}
Jul 10 02:21:22.309291 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:22.309219    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:22.311863 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:21:22.311836    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b998b\" is forbidden: User \"system:node:ip-10-0-132-62.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-62.ec2.internal' and this object" podUID="d2a3cd42-0809-4cef-8d3a-398e078a0e6f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:22.315291 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:22.315276    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:22.317557 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:21:22.317537    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b998b\" is forbidden: User \"system:node:ip-10-0-132-62.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-62.ec2.internal' and this object" podUID="d2a3cd42-0809-4cef-8d3a-398e078a0e6f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:22.436414 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:22.436379    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2a3cd42-0809-4cef-8d3a-398e078a0e6f-extensions-socket-volume\") pod \"d2a3cd42-0809-4cef-8d3a-398e078a0e6f\" (UID: \"d2a3cd42-0809-4cef-8d3a-398e078a0e6f\") "
Jul 10 02:21:22.436566 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:22.436456    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljdzb\" (UniqueName: \"kubernetes.io/projected/d2a3cd42-0809-4cef-8d3a-398e078a0e6f-kube-api-access-ljdzb\") pod \"d2a3cd42-0809-4cef-8d3a-398e078a0e6f\" (UID: \"d2a3cd42-0809-4cef-8d3a-398e078a0e6f\") "
Jul 10 02:21:22.436691 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:22.436649    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2a3cd42-0809-4cef-8d3a-398e078a0e6f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d2a3cd42-0809-4cef-8d3a-398e078a0e6f" (UID: "d2a3cd42-0809-4cef-8d3a-398e078a0e6f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:21:22.438563 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:22.438540    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2a3cd42-0809-4cef-8d3a-398e078a0e6f-kube-api-access-ljdzb" (OuterVolumeSpecName: "kube-api-access-ljdzb") pod "d2a3cd42-0809-4cef-8d3a-398e078a0e6f" (UID: "d2a3cd42-0809-4cef-8d3a-398e078a0e6f"). InnerVolumeSpecName "kube-api-access-ljdzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:21:22.537844 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:22.537814    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d2a3cd42-0809-4cef-8d3a-398e078a0e6f-extensions-socket-volume\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:21:22.537844 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:22.537844    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljdzb\" (UniqueName: \"kubernetes.io/projected/d2a3cd42-0809-4cef-8d3a-398e078a0e6f-kube-api-access-ljdzb\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:21:23.313833 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:23.313803    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:23.317823 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:21:23.317783    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b998b\" is forbidden: User \"system:node:ip-10-0-132-62.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-62.ec2.internal' and this object" podUID="d2a3cd42-0809-4cef-8d3a-398e078a0e6f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:23.324156 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:21:23.324117    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b998b\" is forbidden: User \"system:node:ip-10-0-132-62.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-62.ec2.internal' and this object" podUID="d2a3cd42-0809-4cef-8d3a-398e078a0e6f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:23.564223 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:21:23.564132    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-b998b\" is forbidden: User \"system:node:ip-10-0-132-62.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-62.ec2.internal' and this object" podUID="d2a3cd42-0809-4cef-8d3a-398e078a0e6f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-b998b"
Jul 10 02:21:23.565577 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:23.565541    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2a3cd42-0809-4cef-8d3a-398e078a0e6f" path="/var/lib/kubelet/pods/d2a3cd42-0809-4cef-8d3a-398e078a0e6f/volumes"
Jul 10 02:21:27.331832 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:27.331793    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v" event={"ID":"9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962","Type":"ContainerStarted","Data":"6325ed00161b138c597e2004f6dae31664072eb4847e70edfb928f9c443ba7cf"}
Jul 10 02:21:27.332231 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:27.331870    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"
Jul 10 02:21:27.351956 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:27.351915    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v" podStartSLOduration=1.8562427019999999 podStartE2EDuration="6.351900548s" totalImagesPullingTime="4.495657846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:21:27.351715415 +0000 UTC m=+434.334473101" watchObservedRunningTime="2026-07-10 02:21:27.351900548 +0000 UTC m=+434.334658234"
Jul 10 02:21:38.337314 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:38.337238    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"
Jul 10 02:21:50.880221 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:50.880183    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"]
Jul 10 02:21:50.880688 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:50.880377    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v" podUID="9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962" containerName="manager" containerID="cri-o://6325ed00161b138c597e2004f6dae31664072eb4847e70edfb928f9c443ba7cf" gracePeriod=10
Jul 10 02:21:51.124291 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.124268    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"
Jul 10 02:21:51.159283 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.159218    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962-extensions-socket-volume\") pod \"9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962\" (UID: \"9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962\") "
Jul 10 02:21:51.159283 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.159258    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6k2b\" (UniqueName: \"kubernetes.io/projected/9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962-kube-api-access-v6k2b\") pod \"9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962\" (UID: \"9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962\") "
Jul 10 02:21:51.159726 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.159697    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962" (UID: "9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:21:51.161615 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.161590    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962-kube-api-access-v6k2b" (OuterVolumeSpecName: "kube-api-access-v6k2b") pod "9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962" (UID: "9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962"). InnerVolumeSpecName "kube-api-access-v6k2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:21:51.260656 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.260624    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6k2b\" (UniqueName: \"kubernetes.io/projected/9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962-kube-api-access-v6k2b\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:21:51.260656 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.260649    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962-extensions-socket-volume\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:21:51.422530 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.422422    2579 generic.go:356] "Generic (PLEG): container finished" podID="9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962" containerID="6325ed00161b138c597e2004f6dae31664072eb4847e70edfb928f9c443ba7cf" exitCode=0
Jul 10 02:21:51.422530 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.422473    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v" event={"ID":"9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962","Type":"ContainerDied","Data":"6325ed00161b138c597e2004f6dae31664072eb4847e70edfb928f9c443ba7cf"}
Jul 10 02:21:51.422530 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.422489    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"
Jul 10 02:21:51.422722 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.422578    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v" event={"ID":"9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962","Type":"ContainerDied","Data":"8f0b1b4e254a60b174367cca0d29735edee4157dcd6f500900c2cb0faa4ac631"}
Jul 10 02:21:51.422722 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.422610    2579 scope.go:117] "RemoveContainer" containerID="6325ed00161b138c597e2004f6dae31664072eb4847e70edfb928f9c443ba7cf"
Jul 10 02:21:51.431299 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.431281    2579 scope.go:117] "RemoveContainer" containerID="6325ed00161b138c597e2004f6dae31664072eb4847e70edfb928f9c443ba7cf"
Jul 10 02:21:51.431596 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:21:51.431576    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6325ed00161b138c597e2004f6dae31664072eb4847e70edfb928f9c443ba7cf\": container with ID starting with 6325ed00161b138c597e2004f6dae31664072eb4847e70edfb928f9c443ba7cf not found: ID does not exist" containerID="6325ed00161b138c597e2004f6dae31664072eb4847e70edfb928f9c443ba7cf"
Jul 10 02:21:51.431655 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.431606    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6325ed00161b138c597e2004f6dae31664072eb4847e70edfb928f9c443ba7cf"} err="failed to get container status \"6325ed00161b138c597e2004f6dae31664072eb4847e70edfb928f9c443ba7cf\": rpc error: code = NotFound desc = could not find container \"6325ed00161b138c597e2004f6dae31664072eb4847e70edfb928f9c443ba7cf\": container with ID starting with 6325ed00161b138c597e2004f6dae31664072eb4847e70edfb928f9c443ba7cf not found: ID does not exist"
Jul 10 02:21:51.442848 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.442818    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"]
Jul 10 02:21:51.446685 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.446665    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pkj2v"]
Jul 10 02:21:51.565459 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:21:51.565423    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962" path="/var/lib/kubelet/pods/9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962/volumes"
Jul 10 02:22:08.958930 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:08.958899    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:22:08.959385 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:08.959143    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962" containerName="manager"
Jul 10 02:22:08.959385 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:08.959154    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962" containerName="manager"
Jul 10 02:22:08.959385 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:08.959208    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d5f4b3a-497b-4e3a-8acb-0cf7f2e05962" containerName="manager"
Jul 10 02:22:08.961027 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:08.961011    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"
Jul 10 02:22:08.963356 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:08.963336    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 02:22:08.963468 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:08.963342    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7ttbp\""
Jul 10 02:22:08.964425 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:08.964411    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 02:22:08.964479 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:08.964426    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 10 02:22:08.971754 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:08.971720    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:22:09.004988 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.004963    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:22:09.086220 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.086186    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw9xv\" (UniqueName: \"kubernetes.io/projected/68a64a2d-d11d-4fa1-9bf7-2473052c3d32-kube-api-access-fw9xv\") pod \"limitador-limitador-54d6d988d5-8j9dt\" (UID: \"68a64a2d-d11d-4fa1-9bf7-2473052c3d32\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"
Jul 10 02:22:09.086351 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.086242    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68a64a2d-d11d-4fa1-9bf7-2473052c3d32-config-file\") pod \"limitador-limitador-54d6d988d5-8j9dt\" (UID: \"68a64a2d-d11d-4fa1-9bf7-2473052c3d32\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"
Jul 10 02:22:09.187522 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.187480    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68a64a2d-d11d-4fa1-9bf7-2473052c3d32-config-file\") pod \"limitador-limitador-54d6d988d5-8j9dt\" (UID: \"68a64a2d-d11d-4fa1-9bf7-2473052c3d32\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"
Jul 10 02:22:09.187654 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.187582    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw9xv\" (UniqueName: \"kubernetes.io/projected/68a64a2d-d11d-4fa1-9bf7-2473052c3d32-kube-api-access-fw9xv\") pod \"limitador-limitador-54d6d988d5-8j9dt\" (UID: \"68a64a2d-d11d-4fa1-9bf7-2473052c3d32\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"
Jul 10 02:22:09.188141 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.188118    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/68a64a2d-d11d-4fa1-9bf7-2473052c3d32-config-file\") pod \"limitador-limitador-54d6d988d5-8j9dt\" (UID: \"68a64a2d-d11d-4fa1-9bf7-2473052c3d32\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"
Jul 10 02:22:09.196241 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.196214    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw9xv\" (UniqueName: \"kubernetes.io/projected/68a64a2d-d11d-4fa1-9bf7-2473052c3d32-kube-api-access-fw9xv\") pod \"limitador-limitador-54d6d988d5-8j9dt\" (UID: \"68a64a2d-d11d-4fa1-9bf7-2473052c3d32\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"
Jul 10 02:22:09.271118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.271049    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"
Jul 10 02:22:09.378942 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.378885    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-q2f6q"]
Jul 10 02:22:09.381783 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.381766    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q2f6q"
Jul 10 02:22:09.384216 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.384195    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-78z48\""
Jul 10 02:22:09.389934 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.389909    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-q2f6q"]
Jul 10 02:22:09.397081 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.397062    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:22:09.400179 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:22:09.400154    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a64a2d_d11d_4fa1_9bf7_2473052c3d32.slice/crio-48c27d490192459515eebe51521d9b878dec8f47e1c2f489119522e2d7e887a8 WatchSource:0}: Error finding container 48c27d490192459515eebe51521d9b878dec8f47e1c2f489119522e2d7e887a8: Status 404 returned error can't find the container with id 48c27d490192459515eebe51521d9b878dec8f47e1c2f489119522e2d7e887a8
Jul 10 02:22:09.488964 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.488937    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlwdv\" (UniqueName: \"kubernetes.io/projected/c1de815c-8522-470a-a367-d31dbe1fc269-kube-api-access-qlwdv\") pod \"authorino-58cd955675-q2f6q\" (UID: \"c1de815c-8522-470a-a367-d31dbe1fc269\") " pod="kuadrant-system/authorino-58cd955675-q2f6q"
Jul 10 02:22:09.489868 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.489844    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8j9dt" event={"ID":"68a64a2d-d11d-4fa1-9bf7-2473052c3d32","Type":"ContainerStarted","Data":"48c27d490192459515eebe51521d9b878dec8f47e1c2f489119522e2d7e887a8"}
Jul 10 02:22:09.589953 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.589888    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlwdv\" (UniqueName: \"kubernetes.io/projected/c1de815c-8522-470a-a367-d31dbe1fc269-kube-api-access-qlwdv\") pod \"authorino-58cd955675-q2f6q\" (UID: \"c1de815c-8522-470a-a367-d31dbe1fc269\") " pod="kuadrant-system/authorino-58cd955675-q2f6q"
Jul 10 02:22:09.597495 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.597464    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlwdv\" (UniqueName: \"kubernetes.io/projected/c1de815c-8522-470a-a367-d31dbe1fc269-kube-api-access-qlwdv\") pod \"authorino-58cd955675-q2f6q\" (UID: \"c1de815c-8522-470a-a367-d31dbe1fc269\") " pod="kuadrant-system/authorino-58cd955675-q2f6q"
Jul 10 02:22:09.692158 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.692127    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q2f6q"
Jul 10 02:22:09.819574 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:09.819474    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-q2f6q"]
Jul 10 02:22:09.823300 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:22:09.823271    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1de815c_8522_470a_a367_d31dbe1fc269.slice/crio-d7f442d57f0841c2614f8304ddbeb0fee6272d8933ad0c75775379b4a02842ae WatchSource:0}: Error finding container d7f442d57f0841c2614f8304ddbeb0fee6272d8933ad0c75775379b4a02842ae: Status 404 returned error can't find the container with id d7f442d57f0841c2614f8304ddbeb0fee6272d8933ad0c75775379b4a02842ae
Jul 10 02:22:10.505534 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:10.505439    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q2f6q" event={"ID":"c1de815c-8522-470a-a367-d31dbe1fc269","Type":"ContainerStarted","Data":"d7f442d57f0841c2614f8304ddbeb0fee6272d8933ad0c75775379b4a02842ae"}
Jul 10 02:22:14.067313 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:14.067242    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-q2f6q"]
Jul 10 02:22:14.522413 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:14.522377    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8j9dt" event={"ID":"68a64a2d-d11d-4fa1-9bf7-2473052c3d32","Type":"ContainerStarted","Data":"7c9949de150c6d5433c4aac992054cd7e9a2433a0d41177eb6e69e13c29ae184"}
Jul 10 02:22:14.522636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:14.522467    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"
Jul 10 02:22:14.523788 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:14.523763    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q2f6q" event={"ID":"c1de815c-8522-470a-a367-d31dbe1fc269","Type":"ContainerStarted","Data":"9d92c2f78ce2bf3df1f7eb70c9da7cd77bd16554aa1a21eeee90d03b1f39e729"}
Jul 10 02:22:14.540265 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:14.540219    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-8j9dt" podStartSLOduration=2.216566249 podStartE2EDuration="6.540205036s" totalImagesPullingTime="4.323638787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:22:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:22:14.539625818 +0000 UTC m=+481.522383507" watchObservedRunningTime="2026-07-10 02:22:14.540205036 +0000 UTC m=+481.522962738"
Jul 10 02:22:14.555203 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:14.555164    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-q2f6q" podStartSLOduration=1.664083045 podStartE2EDuration="5.555152511s" totalImagesPullingTime="3.891069466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:22:14.554376229 +0000 UTC m=+481.537133911" watchObservedRunningTime="2026-07-10 02:22:14.555152511 +0000 UTC m=+481.537910197"
Jul 10 02:22:15.527591 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:15.527526    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-q2f6q" podUID="c1de815c-8522-470a-a367-d31dbe1fc269" containerName="authorino" containerID="cri-o://9d92c2f78ce2bf3df1f7eb70c9da7cd77bd16554aa1a21eeee90d03b1f39e729" gracePeriod=30
Jul 10 02:22:15.768630 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:15.768608    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q2f6q"
Jul 10 02:22:15.841117 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:15.841043    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlwdv\" (UniqueName: \"kubernetes.io/projected/c1de815c-8522-470a-a367-d31dbe1fc269-kube-api-access-qlwdv\") pod \"c1de815c-8522-470a-a367-d31dbe1fc269\" (UID: \"c1de815c-8522-470a-a367-d31dbe1fc269\") "
Jul 10 02:22:15.843188 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:15.843156    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1de815c-8522-470a-a367-d31dbe1fc269-kube-api-access-qlwdv" (OuterVolumeSpecName: "kube-api-access-qlwdv") pod "c1de815c-8522-470a-a367-d31dbe1fc269" (UID: "c1de815c-8522-470a-a367-d31dbe1fc269"). InnerVolumeSpecName "kube-api-access-qlwdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:22:15.941637 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:15.941606    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlwdv\" (UniqueName: \"kubernetes.io/projected/c1de815c-8522-470a-a367-d31dbe1fc269-kube-api-access-qlwdv\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:22:16.532378 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:16.532339    2579 generic.go:356] "Generic (PLEG): container finished" podID="c1de815c-8522-470a-a367-d31dbe1fc269" containerID="9d92c2f78ce2bf3df1f7eb70c9da7cd77bd16554aa1a21eeee90d03b1f39e729" exitCode=0
Jul 10 02:22:16.532378 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:16.532382    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q2f6q"
Jul 10 02:22:16.532840 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:16.532423    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q2f6q" event={"ID":"c1de815c-8522-470a-a367-d31dbe1fc269","Type":"ContainerDied","Data":"9d92c2f78ce2bf3df1f7eb70c9da7cd77bd16554aa1a21eeee90d03b1f39e729"}
Jul 10 02:22:16.532840 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:16.532463    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q2f6q" event={"ID":"c1de815c-8522-470a-a367-d31dbe1fc269","Type":"ContainerDied","Data":"d7f442d57f0841c2614f8304ddbeb0fee6272d8933ad0c75775379b4a02842ae"}
Jul 10 02:22:16.532840 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:16.532480    2579 scope.go:117] "RemoveContainer" containerID="9d92c2f78ce2bf3df1f7eb70c9da7cd77bd16554aa1a21eeee90d03b1f39e729"
Jul 10 02:22:16.542026 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:16.541729    2579 scope.go:117] "RemoveContainer" containerID="9d92c2f78ce2bf3df1f7eb70c9da7cd77bd16554aa1a21eeee90d03b1f39e729"
Jul 10 02:22:16.542333 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:22:16.542309    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d92c2f78ce2bf3df1f7eb70c9da7cd77bd16554aa1a21eeee90d03b1f39e729\": container with ID starting with 9d92c2f78ce2bf3df1f7eb70c9da7cd77bd16554aa1a21eeee90d03b1f39e729 not found: ID does not exist" containerID="9d92c2f78ce2bf3df1f7eb70c9da7cd77bd16554aa1a21eeee90d03b1f39e729"
Jul 10 02:22:16.542397 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:16.542340    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d92c2f78ce2bf3df1f7eb70c9da7cd77bd16554aa1a21eeee90d03b1f39e729"} err="failed to get container status \"9d92c2f78ce2bf3df1f7eb70c9da7cd77bd16554aa1a21eeee90d03b1f39e729\": rpc error: code = NotFound desc = could not find container \"9d92c2f78ce2bf3df1f7eb70c9da7cd77bd16554aa1a21eeee90d03b1f39e729\": container with ID starting with 9d92c2f78ce2bf3df1f7eb70c9da7cd77bd16554aa1a21eeee90d03b1f39e729 not found: ID does not exist"
Jul 10 02:22:16.555789 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:16.555771    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-q2f6q"]
Jul 10 02:22:16.557933 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:16.557916    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-q2f6q"]
Jul 10 02:22:17.564995 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:17.564964    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1de815c-8522-470a-a367-d31dbe1fc269" path="/var/lib/kubelet/pods/c1de815c-8522-470a-a367-d31dbe1fc269/volumes"
Jul 10 02:22:25.529057 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:25.529027    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"
Jul 10 02:22:41.764125 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:41.764080    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tlv4c"]
Jul 10 02:22:41.764641 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:41.764346    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1de815c-8522-470a-a367-d31dbe1fc269" containerName="authorino"
Jul 10 02:22:41.764641 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:41.764357    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1de815c-8522-470a-a367-d31dbe1fc269" containerName="authorino"
Jul 10 02:22:41.764641 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:41.764407    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1de815c-8522-470a-a367-d31dbe1fc269" containerName="authorino"
Jul 10 02:22:41.767203 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:41.767187    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tlv4c"
Jul 10 02:22:41.769935 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:41.769916    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-78z48\""
Jul 10 02:22:41.776395 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:41.776363    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tlv4c"]
Jul 10 02:22:41.932451 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:41.932410    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f5nn\" (UniqueName: \"kubernetes.io/projected/970557de-a310-49fb-b855-151650efe263-kube-api-access-2f5nn\") pod \"authorino-5d4b67c446-tlv4c\" (UID: \"970557de-a310-49fb-b855-151650efe263\") " pod="kuadrant-system/authorino-5d4b67c446-tlv4c"
Jul 10 02:22:42.033615 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.033493    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f5nn\" (UniqueName: \"kubernetes.io/projected/970557de-a310-49fb-b855-151650efe263-kube-api-access-2f5nn\") pod \"authorino-5d4b67c446-tlv4c\" (UID: \"970557de-a310-49fb-b855-151650efe263\") " pod="kuadrant-system/authorino-5d4b67c446-tlv4c"
Jul 10 02:22:42.038840 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.038810    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tlv4c"]
Jul 10 02:22:42.039074 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:22:42.039050    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-2f5nn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-tlv4c" podUID="970557de-a310-49fb-b855-151650efe263"
Jul 10 02:22:42.042110 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.042092    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f5nn\" (UniqueName: \"kubernetes.io/projected/970557de-a310-49fb-b855-151650efe263-kube-api-access-2f5nn\") pod \"authorino-5d4b67c446-tlv4c\" (UID: \"970557de-a310-49fb-b855-151650efe263\") " pod="kuadrant-system/authorino-5d4b67c446-tlv4c"
Jul 10 02:22:42.067546 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.067515    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-678974d77d-ssms7"]
Jul 10 02:22:42.070800 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.070783    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-678974d77d-ssms7"
Jul 10 02:22:42.073518 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.073479    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 10 02:22:42.081005 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.080982    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-678974d77d-ssms7"]
Jul 10 02:22:42.234810 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.234770    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/42f7d2a5-bd3c-4533-b3b5-97975373ea3a-tls-cert\") pod \"authorino-678974d77d-ssms7\" (UID: \"42f7d2a5-bd3c-4533-b3b5-97975373ea3a\") " pod="kuadrant-system/authorino-678974d77d-ssms7"
Jul 10 02:22:42.234990 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.234837    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjzhf\" (UniqueName: \"kubernetes.io/projected/42f7d2a5-bd3c-4533-b3b5-97975373ea3a-kube-api-access-jjzhf\") pod \"authorino-678974d77d-ssms7\" (UID: \"42f7d2a5-bd3c-4533-b3b5-97975373ea3a\") " pod="kuadrant-system/authorino-678974d77d-ssms7"
Jul 10 02:22:42.336228 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.336153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/42f7d2a5-bd3c-4533-b3b5-97975373ea3a-tls-cert\") pod \"authorino-678974d77d-ssms7\" (UID: \"42f7d2a5-bd3c-4533-b3b5-97975373ea3a\") " pod="kuadrant-system/authorino-678974d77d-ssms7"
Jul 10 02:22:42.336228 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.336218    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjzhf\" (UniqueName: \"kubernetes.io/projected/42f7d2a5-bd3c-4533-b3b5-97975373ea3a-kube-api-access-jjzhf\") pod \"authorino-678974d77d-ssms7\" (UID: \"42f7d2a5-bd3c-4533-b3b5-97975373ea3a\") " pod="kuadrant-system/authorino-678974d77d-ssms7"
Jul 10 02:22:42.338723 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.338692    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/42f7d2a5-bd3c-4533-b3b5-97975373ea3a-tls-cert\") pod \"authorino-678974d77d-ssms7\" (UID: \"42f7d2a5-bd3c-4533-b3b5-97975373ea3a\") " pod="kuadrant-system/authorino-678974d77d-ssms7"
Jul 10 02:22:42.352623 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.352601    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjzhf\" (UniqueName: \"kubernetes.io/projected/42f7d2a5-bd3c-4533-b3b5-97975373ea3a-kube-api-access-jjzhf\") pod \"authorino-678974d77d-ssms7\" (UID: \"42f7d2a5-bd3c-4533-b3b5-97975373ea3a\") " pod="kuadrant-system/authorino-678974d77d-ssms7"
Jul 10 02:22:42.379710 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.379682    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-678974d77d-ssms7"
Jul 10 02:22:42.511856 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.511811    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-678974d77d-ssms7"]
Jul 10 02:22:42.516487 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:22:42.516461    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42f7d2a5_bd3c_4533_b3b5_97975373ea3a.slice/crio-eb58181c5daa3cd714a456abf42d9e9ffd5d6109995398f6518a08835b8514c9 WatchSource:0}: Error finding container eb58181c5daa3cd714a456abf42d9e9ffd5d6109995398f6518a08835b8514c9: Status 404 returned error can't find the container with id eb58181c5daa3cd714a456abf42d9e9ffd5d6109995398f6518a08835b8514c9
Jul 10 02:22:42.621097 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.621072    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tlv4c"
Jul 10 02:22:42.621097 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.621081    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-678974d77d-ssms7" event={"ID":"42f7d2a5-bd3c-4533-b3b5-97975373ea3a","Type":"ContainerStarted","Data":"eb58181c5daa3cd714a456abf42d9e9ffd5d6109995398f6518a08835b8514c9"}
Jul 10 02:22:42.640842 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.640816    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tlv4c"
Jul 10 02:22:42.739401 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.739371    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2f5nn\" (UniqueName: \"kubernetes.io/projected/970557de-a310-49fb-b855-151650efe263-kube-api-access-2f5nn\") pod \"970557de-a310-49fb-b855-151650efe263\" (UID: \"970557de-a310-49fb-b855-151650efe263\") "
Jul 10 02:22:42.741563 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.741534    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/970557de-a310-49fb-b855-151650efe263-kube-api-access-2f5nn" (OuterVolumeSpecName: "kube-api-access-2f5nn") pod "970557de-a310-49fb-b855-151650efe263" (UID: "970557de-a310-49fb-b855-151650efe263"). InnerVolumeSpecName "kube-api-access-2f5nn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:22:42.840360 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:42.840331    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2f5nn\" (UniqueName: \"kubernetes.io/projected/970557de-a310-49fb-b855-151650efe263-kube-api-access-2f5nn\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:22:43.626454 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:43.626415    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-678974d77d-ssms7" event={"ID":"42f7d2a5-bd3c-4533-b3b5-97975373ea3a","Type":"ContainerStarted","Data":"03e5a653a0a48969165f19d69e7fdc5f29b131b8f7b3152a93ebe106dfc263a8"}
Jul 10 02:22:43.626622 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:43.626479    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tlv4c"
Jul 10 02:22:43.643111 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:43.643053    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-678974d77d-ssms7" podStartSLOduration=1.032942422 podStartE2EDuration="1.643039823s" totalImagesPullingTime="610.097401ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:22:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:22:43.642224303 +0000 UTC m=+510.624981989" watchObservedRunningTime="2026-07-10 02:22:43.643039823 +0000 UTC m=+510.625797509"
Jul 10 02:22:43.669719 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:43.669691    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tlv4c"]
Jul 10 02:22:43.673549 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:43.673524    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tlv4c"]
Jul 10 02:22:45.569388 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:22:45.569351    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="970557de-a310-49fb-b855-151650efe263" path="/var/lib/kubelet/pods/970557de-a310-49fb-b855-151650efe263/volumes"
Jul 10 02:23:23.667783 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:23:23.667753    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:23:42.106387 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:23:42.106325    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:23:56.530663 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:23:56.530627    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:24:07.670988 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:07.670952    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:24:15.174990 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:15.174957    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:24:22.278764 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:22.278730    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:24:29.369120 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.369082    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"]
Jul 10 02:24:29.371702 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.371685    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.375038 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.375013    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 10 02:24:29.376249 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.376231    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 10 02:24:29.376365 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.376236    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 10 02:24:29.376365 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.376331    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-g7bnd\""
Jul 10 02:24:29.382455 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.382430    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"]
Jul 10 02:24:29.452316 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.452282    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5000069-9d9e-4358-b68c-c7489c292b12-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.452606 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.452588    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5000069-9d9e-4358-b68c-c7489c292b12-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.452745 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.452731    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5000069-9d9e-4358-b68c-c7489c292b12-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.452851 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.452836    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5000069-9d9e-4358-b68c-c7489c292b12-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.452955 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.452943    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5000069-9d9e-4358-b68c-c7489c292b12-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.453081 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.453067    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gktj\" (UniqueName: \"kubernetes.io/projected/e5000069-9d9e-4358-b68c-c7489c292b12-kube-api-access-6gktj\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.554216 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.554186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5000069-9d9e-4358-b68c-c7489c292b12-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.554216 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.554220    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5000069-9d9e-4358-b68c-c7489c292b12-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.554440 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.554237    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5000069-9d9e-4358-b68c-c7489c292b12-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.554440 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.554256    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5000069-9d9e-4358-b68c-c7489c292b12-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.554440 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.554288    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gktj\" (UniqueName: \"kubernetes.io/projected/e5000069-9d9e-4358-b68c-c7489c292b12-kube-api-access-6gktj\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.554440 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.554331    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5000069-9d9e-4358-b68c-c7489c292b12-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.554713 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.554688    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5000069-9d9e-4358-b68c-c7489c292b12-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.554778 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.554698    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5000069-9d9e-4358-b68c-c7489c292b12-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.554778 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.554736    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5000069-9d9e-4358-b68c-c7489c292b12-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.556761 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.556744    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5000069-9d9e-4358-b68c-c7489c292b12-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.556936 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.556920    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5000069-9d9e-4358-b68c-c7489c292b12-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.563699 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.563677    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gktj\" (UniqueName: \"kubernetes.io/projected/e5000069-9d9e-4358-b68c-c7489c292b12-kube-api-access-6gktj\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7\" (UID: \"e5000069-9d9e-4358-b68c-c7489c292b12\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.681414 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.681323    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:29.812954 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.812929    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"]
Jul 10 02:24:29.815413 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:24:29.815381    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5000069_9d9e_4358_b68c_c7489c292b12.slice/crio-1d2c21be69693a468a2fe0d477bdba1ce2545818861d0095fce04696522fbd72 WatchSource:0}: Error finding container 1d2c21be69693a468a2fe0d477bdba1ce2545818861d0095fce04696522fbd72: Status 404 returned error can't find the container with id 1d2c21be69693a468a2fe0d477bdba1ce2545818861d0095fce04696522fbd72
Jul 10 02:24:29.816605 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:29.816590    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:24:30.021684 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:30.021579    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7" event={"ID":"e5000069-9d9e-4358-b68c-c7489c292b12","Type":"ContainerStarted","Data":"1d2c21be69693a468a2fe0d477bdba1ce2545818861d0095fce04696522fbd72"}
Jul 10 02:24:31.027052 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:31.026963    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7" event={"ID":"e5000069-9d9e-4358-b68c-c7489c292b12","Type":"ContainerStarted","Data":"0c89b92267271119e239379b195e2c88f0cbfe8e2e92bb9cf7880daa56347b7d"}
Jul 10 02:24:31.047192 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:31.047143    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7" podStartSLOduration=1.092559307 podStartE2EDuration="2.047127185s" totalImagesPullingTime="954.567878ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:24:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:24:31.045819065 +0000 UTC m=+618.028576750" watchObservedRunningTime="2026-07-10 02:24:31.047127185 +0000 UTC m=+618.029884872"
Jul 10 02:24:31.969495 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:31.969454    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:24:39.682202 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:39.682106    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:39.682202 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:39.682176    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:39.694553 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:39.694526    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:24:40.072283 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:24:40.072188    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7"
Jul 10 02:25:51.126879 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:51.126843    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-77fd4fbdbc-2z4nv"]
Jul 10 02:25:51.129760 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:51.129744    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77fd4fbdbc-2z4nv"
Jul 10 02:25:51.140650 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:51.140618    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-77fd4fbdbc-2z4nv"]
Jul 10 02:25:51.194723 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:51.194685    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n47cr\" (UniqueName: \"kubernetes.io/projected/38d7910d-0e56-47b8-ac09-155be82b3034-kube-api-access-n47cr\") pod \"authorino-77fd4fbdbc-2z4nv\" (UID: \"38d7910d-0e56-47b8-ac09-155be82b3034\") " pod="kuadrant-system/authorino-77fd4fbdbc-2z4nv"
Jul 10 02:25:51.194887 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:51.194731    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38d7910d-0e56-47b8-ac09-155be82b3034-tls-cert\") pod \"authorino-77fd4fbdbc-2z4nv\" (UID: \"38d7910d-0e56-47b8-ac09-155be82b3034\") " pod="kuadrant-system/authorino-77fd4fbdbc-2z4nv"
Jul 10 02:25:51.295964 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:51.295925    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38d7910d-0e56-47b8-ac09-155be82b3034-tls-cert\") pod \"authorino-77fd4fbdbc-2z4nv\" (UID: \"38d7910d-0e56-47b8-ac09-155be82b3034\") " pod="kuadrant-system/authorino-77fd4fbdbc-2z4nv"
Jul 10 02:25:51.296153 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:51.296022    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n47cr\" (UniqueName: \"kubernetes.io/projected/38d7910d-0e56-47b8-ac09-155be82b3034-kube-api-access-n47cr\") pod \"authorino-77fd4fbdbc-2z4nv\" (UID: \"38d7910d-0e56-47b8-ac09-155be82b3034\") " pod="kuadrant-system/authorino-77fd4fbdbc-2z4nv"
Jul 10 02:25:51.298557 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:51.298533    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38d7910d-0e56-47b8-ac09-155be82b3034-tls-cert\") pod \"authorino-77fd4fbdbc-2z4nv\" (UID: \"38d7910d-0e56-47b8-ac09-155be82b3034\") " pod="kuadrant-system/authorino-77fd4fbdbc-2z4nv"
Jul 10 02:25:51.304603 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:51.304571    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n47cr\" (UniqueName: \"kubernetes.io/projected/38d7910d-0e56-47b8-ac09-155be82b3034-kube-api-access-n47cr\") pod \"authorino-77fd4fbdbc-2z4nv\" (UID: \"38d7910d-0e56-47b8-ac09-155be82b3034\") " pod="kuadrant-system/authorino-77fd4fbdbc-2z4nv"
Jul 10 02:25:51.438696 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:51.438612    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77fd4fbdbc-2z4nv"
Jul 10 02:25:51.569792 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:51.569769    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-77fd4fbdbc-2z4nv"]
Jul 10 02:25:51.571803 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:25:51.571774    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38d7910d_0e56_47b8_ac09_155be82b3034.slice/crio-934b2dc5ae23f384658b1a9c529e9af07d23d14a3a1175aaa1826b112e86df08 WatchSource:0}: Error finding container 934b2dc5ae23f384658b1a9c529e9af07d23d14a3a1175aaa1826b112e86df08: Status 404 returned error can't find the container with id 934b2dc5ae23f384658b1a9c529e9af07d23d14a3a1175aaa1826b112e86df08
Jul 10 02:25:52.343573 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:52.343458    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77fd4fbdbc-2z4nv" event={"ID":"38d7910d-0e56-47b8-ac09-155be82b3034","Type":"ContainerStarted","Data":"f986ed32d2c1fa40dbf9a148e57b138ad1d19d50d2794ca64a46b7a95f411932"}
Jul 10 02:25:52.343573 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:52.343531    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77fd4fbdbc-2z4nv" event={"ID":"38d7910d-0e56-47b8-ac09-155be82b3034","Type":"ContainerStarted","Data":"934b2dc5ae23f384658b1a9c529e9af07d23d14a3a1175aaa1826b112e86df08"}
Jul 10 02:25:52.366952 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:52.366897    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-77fd4fbdbc-2z4nv" podStartSLOduration=0.848041819 podStartE2EDuration="1.366880164s" totalImagesPullingTime="518.838345ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:25:52.365972106 +0000 UTC m=+699.348729792" watchObservedRunningTime="2026-07-10 02:25:52.366880164 +0000 UTC m=+699.349637850"
Jul 10 02:25:52.396374 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:52.396344    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-678974d77d-ssms7"]
Jul 10 02:25:52.396572 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:52.396550    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-678974d77d-ssms7" podUID="42f7d2a5-bd3c-4533-b3b5-97975373ea3a" containerName="authorino" containerID="cri-o://03e5a653a0a48969165f19d69e7fdc5f29b131b8f7b3152a93ebe106dfc263a8" gracePeriod=30
Jul 10 02:25:52.639801 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:52.639748    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-678974d77d-ssms7"
Jul 10 02:25:52.707849 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:52.707807    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjzhf\" (UniqueName: \"kubernetes.io/projected/42f7d2a5-bd3c-4533-b3b5-97975373ea3a-kube-api-access-jjzhf\") pod \"42f7d2a5-bd3c-4533-b3b5-97975373ea3a\" (UID: \"42f7d2a5-bd3c-4533-b3b5-97975373ea3a\") "
Jul 10 02:25:52.708041 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:52.707867    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/42f7d2a5-bd3c-4533-b3b5-97975373ea3a-tls-cert\") pod \"42f7d2a5-bd3c-4533-b3b5-97975373ea3a\" (UID: \"42f7d2a5-bd3c-4533-b3b5-97975373ea3a\") "
Jul 10 02:25:52.710293 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:52.710256    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42f7d2a5-bd3c-4533-b3b5-97975373ea3a-kube-api-access-jjzhf" (OuterVolumeSpecName: "kube-api-access-jjzhf") pod "42f7d2a5-bd3c-4533-b3b5-97975373ea3a" (UID: "42f7d2a5-bd3c-4533-b3b5-97975373ea3a"). InnerVolumeSpecName "kube-api-access-jjzhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:25:52.718009 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:52.717979    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42f7d2a5-bd3c-4533-b3b5-97975373ea3a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "42f7d2a5-bd3c-4533-b3b5-97975373ea3a" (UID: "42f7d2a5-bd3c-4533-b3b5-97975373ea3a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:25:52.809274 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:52.809237    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/42f7d2a5-bd3c-4533-b3b5-97975373ea3a-tls-cert\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:25:52.809274 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:52.809266    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjzhf\" (UniqueName: \"kubernetes.io/projected/42f7d2a5-bd3c-4533-b3b5-97975373ea3a-kube-api-access-jjzhf\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:25:53.349307 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:53.349269    2579 generic.go:356] "Generic (PLEG): container finished" podID="42f7d2a5-bd3c-4533-b3b5-97975373ea3a" containerID="03e5a653a0a48969165f19d69e7fdc5f29b131b8f7b3152a93ebe106dfc263a8" exitCode=0
Jul 10 02:25:53.349789 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:53.349319    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-678974d77d-ssms7"
Jul 10 02:25:53.349789 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:53.349360    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-678974d77d-ssms7" event={"ID":"42f7d2a5-bd3c-4533-b3b5-97975373ea3a","Type":"ContainerDied","Data":"03e5a653a0a48969165f19d69e7fdc5f29b131b8f7b3152a93ebe106dfc263a8"}
Jul 10 02:25:53.349789 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:53.349417    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-678974d77d-ssms7" event={"ID":"42f7d2a5-bd3c-4533-b3b5-97975373ea3a","Type":"ContainerDied","Data":"eb58181c5daa3cd714a456abf42d9e9ffd5d6109995398f6518a08835b8514c9"}
Jul 10 02:25:53.349789 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:53.349441    2579 scope.go:117] "RemoveContainer" containerID="03e5a653a0a48969165f19d69e7fdc5f29b131b8f7b3152a93ebe106dfc263a8"
Jul 10 02:25:53.358886 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:53.358860    2579 scope.go:117] "RemoveContainer" containerID="03e5a653a0a48969165f19d69e7fdc5f29b131b8f7b3152a93ebe106dfc263a8"
Jul 10 02:25:53.359164 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:25:53.359142    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03e5a653a0a48969165f19d69e7fdc5f29b131b8f7b3152a93ebe106dfc263a8\": container with ID starting with 03e5a653a0a48969165f19d69e7fdc5f29b131b8f7b3152a93ebe106dfc263a8 not found: ID does not exist" containerID="03e5a653a0a48969165f19d69e7fdc5f29b131b8f7b3152a93ebe106dfc263a8"
Jul 10 02:25:53.359249 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:53.359173    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03e5a653a0a48969165f19d69e7fdc5f29b131b8f7b3152a93ebe106dfc263a8"} err="failed to get container status \"03e5a653a0a48969165f19d69e7fdc5f29b131b8f7b3152a93ebe106dfc263a8\": rpc error: code = NotFound desc = could not find container \"03e5a653a0a48969165f19d69e7fdc5f29b131b8f7b3152a93ebe106dfc263a8\": container with ID starting with 03e5a653a0a48969165f19d69e7fdc5f29b131b8f7b3152a93ebe106dfc263a8 not found: ID does not exist"
Jul 10 02:25:53.374643 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:53.374571    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-678974d77d-ssms7"]
Jul 10 02:25:53.379088 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:53.378964    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-678974d77d-ssms7"]
Jul 10 02:25:53.566408 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:25:53.566339    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42f7d2a5-bd3c-4533-b3b5-97975373ea3a" path="/var/lib/kubelet/pods/42f7d2a5-bd3c-4533-b3b5-97975373ea3a/volumes"
Jul 10 02:26:20.059381 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:26:20.059329    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:26:29.958080 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:26:29.958040    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:26:40.460905 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:26:40.460874    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:26:50.969357 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:26:50.969313    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:27:01.360053 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:27:01.360021    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:27:27.874476 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:27:27.874429    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:27:39.667567 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:27:39.667477    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:28:18.111253 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:28:18.111219    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:28:25.867615 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:28:25.867579    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:28:36.375245 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:28:36.375208    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:28:42.279677 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:28:42.279640    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:28:53.456952 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:28:53.456917    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:29:05.459529 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:29:05.459428    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:29:16.373892 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:29:16.373857    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:29:26.158608 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:29:26.158570    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:29:48.267515 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:29:48.267467    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:29:57.755766 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:29:57.755726    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:30:20.671198 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:30:20.671164    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:30:31.262337 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:30:31.262300    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:30:47.466458 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:30:47.466380    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:30:57.764906 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:30:57.764866    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:31:08.781090 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:31:08.781050    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:31:18.162548 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:31:18.162496    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:31:27.363580 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:31:27.363540    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:33:15.566166 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:33:15.566135    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:33:24.767098 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:33:24.767065    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:33:38.180422 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:33:38.180325    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:33:49.367465 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:33:49.367431    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:34:00.282734 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:34:00.282697    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:34:11.476108 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:34:11.476073    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:34:20.167068 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:34:20.167032    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:34:31.363210 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:34:31.363174    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:34:42.086228 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:34:42.086193    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:34:51.169100 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:34:51.169061    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:35:02.068225 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:35:02.068187    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:35:24.078960 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:35:24.078880    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:35:33.371912 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:35:33.371880    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:35:43.471483 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:35:43.471449    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:35:53.487710 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:35:53.487680    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:36:00.461010 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:36:00.460980    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:36:15.975070 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:36:15.975039    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:36:27.769418 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:36:27.769368    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:36:37.278227 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:36:37.278149    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:36:45.272533 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:36:45.271973    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:36:55.978218 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:36:55.978179    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:37:05.463827 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:37:05.463795    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:37:18.865641 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:37:18.865608    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:37:28.270456 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:37:28.270420    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:37:37.763247 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:37:37.763210    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:37:47.374888 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:37:47.374852    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:38:00.678315 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:38:00.678275    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:38:09.958779 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:38:09.958688    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:38:21.861137 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:38:21.861101    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:38:33.672786 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:38:33.672749    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:38:43.066058 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:38:43.066028    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:38:56.863586 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:38:56.863551    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:39:05.973314 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:05.973280    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:39:15.762513 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:15.762467    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:39:17.689224 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.689195    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"]
Jul 10 02:39:17.689622 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.689496    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42f7d2a5-bd3c-4533-b3b5-97975373ea3a" containerName="authorino"
Jul 10 02:39:17.689622 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.689531    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="42f7d2a5-bd3c-4533-b3b5-97975373ea3a" containerName="authorino"
Jul 10 02:39:17.689622 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.689591    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="42f7d2a5-bd3c-4533-b3b5-97975373ea3a" containerName="authorino"
Jul 10 02:39:17.692346 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.692329    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"
Jul 10 02:39:17.697641 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.697620    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-32ad1e23-serving-cert\""
Jul 10 02:39:17.697982 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.697962    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-scvcw\""
Jul 10 02:39:17.699026 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.698998    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 02:39:17.699110 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.698997    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 02:39:17.714030 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.714008    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"]
Jul 10 02:39:17.792375 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.792340    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b45ba4b-a600-4ec7-a281-4879b019d14d-openshift-service-ca\") pod \"maas-api-e2e-ait-32ad1e23-c958775c7-hrd54\" (UID: \"6b45ba4b-a600-4ec7-a281-4879b019d14d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"
Jul 10 02:39:17.792557 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.792393    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b45ba4b-a600-4ec7-a281-4879b019d14d-maas-api-tls\") pod \"maas-api-e2e-ait-32ad1e23-c958775c7-hrd54\" (UID: \"6b45ba4b-a600-4ec7-a281-4879b019d14d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"
Jul 10 02:39:17.792557 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.792468    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l2v9\" (UniqueName: \"kubernetes.io/projected/6b45ba4b-a600-4ec7-a281-4879b019d14d-kube-api-access-4l2v9\") pod \"maas-api-e2e-ait-32ad1e23-c958775c7-hrd54\" (UID: \"6b45ba4b-a600-4ec7-a281-4879b019d14d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"
Jul 10 02:39:17.893423 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.893390    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b45ba4b-a600-4ec7-a281-4879b019d14d-openshift-service-ca\") pod \"maas-api-e2e-ait-32ad1e23-c958775c7-hrd54\" (UID: \"6b45ba4b-a600-4ec7-a281-4879b019d14d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"
Jul 10 02:39:17.893584 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.893450    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b45ba4b-a600-4ec7-a281-4879b019d14d-maas-api-tls\") pod \"maas-api-e2e-ait-32ad1e23-c958775c7-hrd54\" (UID: \"6b45ba4b-a600-4ec7-a281-4879b019d14d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"
Jul 10 02:39:17.893584 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.893481    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l2v9\" (UniqueName: \"kubernetes.io/projected/6b45ba4b-a600-4ec7-a281-4879b019d14d-kube-api-access-4l2v9\") pod \"maas-api-e2e-ait-32ad1e23-c958775c7-hrd54\" (UID: \"6b45ba4b-a600-4ec7-a281-4879b019d14d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"
Jul 10 02:39:17.893672 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:39:17.893616    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-serving-cert: secret "maas-api-e2e-ait-32ad1e23-serving-cert" not found
Jul 10 02:39:17.893712 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:39:17.893687    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b45ba4b-a600-4ec7-a281-4879b019d14d-maas-api-tls podName:6b45ba4b-a600-4ec7-a281-4879b019d14d nodeName:}" failed. No retries permitted until 2026-07-10 02:39:18.39367124 +0000 UTC m=+1505.376428913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6b45ba4b-a600-4ec7-a281-4879b019d14d-maas-api-tls") pod "maas-api-e2e-ait-32ad1e23-c958775c7-hrd54" (UID: "6b45ba4b-a600-4ec7-a281-4879b019d14d") : secret "maas-api-e2e-ait-32ad1e23-serving-cert" not found
Jul 10 02:39:17.894085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.894068    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b45ba4b-a600-4ec7-a281-4879b019d14d-openshift-service-ca\") pod \"maas-api-e2e-ait-32ad1e23-c958775c7-hrd54\" (UID: \"6b45ba4b-a600-4ec7-a281-4879b019d14d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"
Jul 10 02:39:17.912599 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:17.912573    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l2v9\" (UniqueName: \"kubernetes.io/projected/6b45ba4b-a600-4ec7-a281-4879b019d14d-kube-api-access-4l2v9\") pod \"maas-api-e2e-ait-32ad1e23-c958775c7-hrd54\" (UID: \"6b45ba4b-a600-4ec7-a281-4879b019d14d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"
Jul 10 02:39:18.397231 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:18.397201    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b45ba4b-a600-4ec7-a281-4879b019d14d-maas-api-tls\") pod \"maas-api-e2e-ait-32ad1e23-c958775c7-hrd54\" (UID: \"6b45ba4b-a600-4ec7-a281-4879b019d14d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"
Jul 10 02:39:18.399686 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:18.399667    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b45ba4b-a600-4ec7-a281-4879b019d14d-maas-api-tls\") pod \"maas-api-e2e-ait-32ad1e23-c958775c7-hrd54\" (UID: \"6b45ba4b-a600-4ec7-a281-4879b019d14d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"
Jul 10 02:39:18.601986 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:18.601958    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"
Jul 10 02:39:18.744349 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:18.744318    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"]
Jul 10 02:39:18.747441 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:39:18.747410    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b45ba4b_a600_4ec7_a281_4879b019d14d.slice/crio-53c62a0b0fdff147bf23a360bb3617501c6972e60225fcdda8ac7e81f0e5c913 WatchSource:0}: Error finding container 53c62a0b0fdff147bf23a360bb3617501c6972e60225fcdda8ac7e81f0e5c913: Status 404 returned error can't find the container with id 53c62a0b0fdff147bf23a360bb3617501c6972e60225fcdda8ac7e81f0e5c913
Jul 10 02:39:18.748749 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:18.748731    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:39:19.459154 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:19.459120    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54" event={"ID":"6b45ba4b-a600-4ec7-a281-4879b019d14d","Type":"ContainerStarted","Data":"53c62a0b0fdff147bf23a360bb3617501c6972e60225fcdda8ac7e81f0e5c913"}
Jul 10 02:39:21.468191 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:21.468087    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54" event={"ID":"6b45ba4b-a600-4ec7-a281-4879b019d14d","Type":"ContainerStarted","Data":"4a6a62a2fc9c8638ff5dfac3736fb17d76dbf109daa7776b389a968388f23273"}
Jul 10 02:39:21.468606 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:21.468297    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"
Jul 10 02:39:21.487423 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:21.487372    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54" podStartSLOduration=2.083066689 podStartE2EDuration="4.487357759s" totalImagesPullingTime="2.40429107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:39:21.486881193 +0000 UTC m=+1508.469638880" watchObservedRunningTime="2026-07-10 02:39:21.487357759 +0000 UTC m=+1508.470115493"
Jul 10 02:39:23.105820 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:23.105792    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"]
Jul 10 02:39:23.477619 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:23.477499    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54" podUID="6b45ba4b-a600-4ec7-a281-4879b019d14d" containerName="maas-api" containerID="cri-o://4a6a62a2fc9c8638ff5dfac3736fb17d76dbf109daa7776b389a968388f23273" gracePeriod=30
Jul 10 02:39:23.719018 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:23.718987    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"
Jul 10 02:39:23.740785 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:23.740711    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b45ba4b-a600-4ec7-a281-4879b019d14d-maas-api-tls\") pod \"6b45ba4b-a600-4ec7-a281-4879b019d14d\" (UID: \"6b45ba4b-a600-4ec7-a281-4879b019d14d\") "
Jul 10 02:39:23.740785 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:23.740764    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b45ba4b-a600-4ec7-a281-4879b019d14d-openshift-service-ca\") pod \"6b45ba4b-a600-4ec7-a281-4879b019d14d\" (UID: \"6b45ba4b-a600-4ec7-a281-4879b019d14d\") "
Jul 10 02:39:23.740947 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:23.740802    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4l2v9\" (UniqueName: \"kubernetes.io/projected/6b45ba4b-a600-4ec7-a281-4879b019d14d-kube-api-access-4l2v9\") pod \"6b45ba4b-a600-4ec7-a281-4879b019d14d\" (UID: \"6b45ba4b-a600-4ec7-a281-4879b019d14d\") "
Jul 10 02:39:23.741109 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:23.741084    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b45ba4b-a600-4ec7-a281-4879b019d14d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6b45ba4b-a600-4ec7-a281-4879b019d14d" (UID: "6b45ba4b-a600-4ec7-a281-4879b019d14d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:39:23.743216 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:23.743190    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b45ba4b-a600-4ec7-a281-4879b019d14d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6b45ba4b-a600-4ec7-a281-4879b019d14d" (UID: "6b45ba4b-a600-4ec7-a281-4879b019d14d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:39:23.743216 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:23.743201    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b45ba4b-a600-4ec7-a281-4879b019d14d-kube-api-access-4l2v9" (OuterVolumeSpecName: "kube-api-access-4l2v9") pod "6b45ba4b-a600-4ec7-a281-4879b019d14d" (UID: "6b45ba4b-a600-4ec7-a281-4879b019d14d"). InnerVolumeSpecName "kube-api-access-4l2v9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:39:23.841559 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:23.841526    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b45ba4b-a600-4ec7-a281-4879b019d14d-maas-api-tls\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:39:23.841559 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:23.841550    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b45ba4b-a600-4ec7-a281-4879b019d14d-openshift-service-ca\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:39:23.841559 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:23.841559    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l2v9\" (UniqueName: \"kubernetes.io/projected/6b45ba4b-a600-4ec7-a281-4879b019d14d-kube-api-access-4l2v9\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:39:24.481996 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:24.481957    2579 generic.go:356] "Generic (PLEG): container finished" podID="6b45ba4b-a600-4ec7-a281-4879b019d14d" containerID="4a6a62a2fc9c8638ff5dfac3736fb17d76dbf109daa7776b389a968388f23273" exitCode=0
Jul 10 02:39:24.481996 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:24.481997    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"
Jul 10 02:39:24.482440 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:24.482042    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54" event={"ID":"6b45ba4b-a600-4ec7-a281-4879b019d14d","Type":"ContainerDied","Data":"4a6a62a2fc9c8638ff5dfac3736fb17d76dbf109daa7776b389a968388f23273"}
Jul 10 02:39:24.482440 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:24.482090    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54" event={"ID":"6b45ba4b-a600-4ec7-a281-4879b019d14d","Type":"ContainerDied","Data":"53c62a0b0fdff147bf23a360bb3617501c6972e60225fcdda8ac7e81f0e5c913"}
Jul 10 02:39:24.482440 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:24.482110    2579 scope.go:117] "RemoveContainer" containerID="4a6a62a2fc9c8638ff5dfac3736fb17d76dbf109daa7776b389a968388f23273"
Jul 10 02:39:24.496679 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:24.496663    2579 scope.go:117] "RemoveContainer" containerID="4a6a62a2fc9c8638ff5dfac3736fb17d76dbf109daa7776b389a968388f23273"
Jul 10 02:39:24.496965 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:39:24.496943    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a6a62a2fc9c8638ff5dfac3736fb17d76dbf109daa7776b389a968388f23273\": container with ID starting with 4a6a62a2fc9c8638ff5dfac3736fb17d76dbf109daa7776b389a968388f23273 not found: ID does not exist" containerID="4a6a62a2fc9c8638ff5dfac3736fb17d76dbf109daa7776b389a968388f23273"
Jul 10 02:39:24.497027 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:24.496974    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a6a62a2fc9c8638ff5dfac3736fb17d76dbf109daa7776b389a968388f23273"} err="failed to get container status \"4a6a62a2fc9c8638ff5dfac3736fb17d76dbf109daa7776b389a968388f23273\": rpc error: code = NotFound desc = could not find container \"4a6a62a2fc9c8638ff5dfac3736fb17d76dbf109daa7776b389a968388f23273\": container with ID starting with 4a6a62a2fc9c8638ff5dfac3736fb17d76dbf109daa7776b389a968388f23273 not found: ID does not exist"
Jul 10 02:39:24.506262 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:24.506236    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"]
Jul 10 02:39:24.512541 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:24.512521    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-32ad1e23-c958775c7-hrd54"]
Jul 10 02:39:25.565590 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:25.565556    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b45ba4b-a600-4ec7-a281-4879b019d14d" path="/var/lib/kubelet/pods/6b45ba4b-a600-4ec7-a281-4879b019d14d/volumes"
Jul 10 02:39:32.957122 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:32.957093    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:39:44.052748 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:44.052713    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:39:53.657092 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:39:53.657059    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:40:04.708117 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:40:04.708063    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:41:24.452240 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:24.452155    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:41:33.662423 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:33.662388    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:41:40.715041 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.715008    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"]
Jul 10 02:41:40.715432 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.715348    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b45ba4b-a600-4ec7-a281-4879b019d14d" containerName="maas-api"
Jul 10 02:41:40.715432 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.715364    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b45ba4b-a600-4ec7-a281-4879b019d14d" containerName="maas-api"
Jul 10 02:41:40.715432 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.715422    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b45ba4b-a600-4ec7-a281-4879b019d14d" containerName="maas-api"
Jul 10 02:41:40.718306 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.718291    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:40.721563 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.721539    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-scvcw\""
Jul 10 02:41:40.721791 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.721775    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-57470f4b-serving-cert\""
Jul 10 02:41:40.722696 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.722677    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 02:41:40.722786 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.722705    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 02:41:40.731961 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.731936    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"]
Jul 10 02:41:40.792135 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.792096    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-openshift-service-ca\") pod \"maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x\" (UID: \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:40.792291 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.792150    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-maas-api-tls\") pod \"maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x\" (UID: \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:40.792291 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.792229    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcfzp\" (UniqueName: \"kubernetes.io/projected/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-kube-api-access-gcfzp\") pod \"maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x\" (UID: \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:40.892796 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.892758    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-maas-api-tls\") pod \"maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x\" (UID: \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:40.892987 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.892805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcfzp\" (UniqueName: \"kubernetes.io/projected/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-kube-api-access-gcfzp\") pod \"maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x\" (UID: \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:40.892987 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.892853    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-openshift-service-ca\") pod \"maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x\" (UID: \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:40.892987 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:41:40.892913    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-serving-cert: secret "maas-api-e2e-mt-57470f4b-serving-cert" not found
Jul 10 02:41:40.892987 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:41:40.892982    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-maas-api-tls podName:cdc66edf-cdaf-428c-8c6d-34d8cd9aa444 nodeName:}" failed. No retries permitted until 2026-07-10 02:41:41.392964633 +0000 UTC m=+1648.375722297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-maas-api-tls") pod "maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x" (UID: "cdc66edf-cdaf-428c-8c6d-34d8cd9aa444") : secret "maas-api-e2e-mt-57470f4b-serving-cert" not found
Jul 10 02:41:40.893461 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.893443    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-openshift-service-ca\") pod \"maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x\" (UID: \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:40.901867 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:40.901846    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcfzp\" (UniqueName: \"kubernetes.io/projected/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-kube-api-access-gcfzp\") pod \"maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x\" (UID: \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:41.395938 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:41.395897    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-maas-api-tls\") pod \"maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x\" (UID: \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:41.398557 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:41.398535    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-maas-api-tls\") pod \"maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x\" (UID: \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:41.628717 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:41.628681    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:41.758249 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:41.758220    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"]
Jul 10 02:41:41.761556 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:41:41.761525    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdc66edf_cdaf_428c_8c6d_34d8cd9aa444.slice/crio-c14359c033245b92ca314b7b9f440cae7db3968373c436075cdfa022103b113d WatchSource:0}: Error finding container c14359c033245b92ca314b7b9f440cae7db3968373c436075cdfa022103b113d: Status 404 returned error can't find the container with id c14359c033245b92ca314b7b9f440cae7db3968373c436075cdfa022103b113d
Jul 10 02:41:42.007388 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:42.007307    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x" event={"ID":"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444","Type":"ContainerStarted","Data":"c14359c033245b92ca314b7b9f440cae7db3968373c436075cdfa022103b113d"}
Jul 10 02:41:43.012782 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:43.012749    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x" event={"ID":"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444","Type":"ContainerStarted","Data":"b3a9b1fb4046ae2e607efe93fa398dbd0a6c34400c95d4090097c8368b420543"}
Jul 10 02:41:43.013184 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:43.012858    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:43.031680 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:43.031638    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x" podStartSLOduration=2.559947255 podStartE2EDuration="3.031623835s" totalImagesPullingTime="471.67658ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:43.030796859 +0000 UTC m=+1650.013554546" watchObservedRunningTime="2026-07-10 02:41:43.031623835 +0000 UTC m=+1650.014381521"
Jul 10 02:41:46.161156 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:46.161124    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:41:49.021664 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:49.021636    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:53.360652 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:53.360612    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"]
Jul 10 02:41:53.361483 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:53.361427    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x" podUID="cdc66edf-cdaf-428c-8c6d-34d8cd9aa444" containerName="maas-api" containerID="cri-o://b3a9b1fb4046ae2e607efe93fa398dbd0a6c34400c95d4090097c8368b420543" gracePeriod=30
Jul 10 02:41:53.602937 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:53.602915    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:53.695706 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:53.695640    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-maas-api-tls\") pod \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\" (UID: \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\") "
Jul 10 02:41:53.695834 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:53.695714    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcfzp\" (UniqueName: \"kubernetes.io/projected/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-kube-api-access-gcfzp\") pod \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\" (UID: \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\") "
Jul 10 02:41:53.695834 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:53.695734    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-openshift-service-ca\") pod \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\" (UID: \"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444\") "
Jul 10 02:41:53.696129 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:53.696099    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cdc66edf-cdaf-428c-8c6d-34d8cd9aa444" (UID: "cdc66edf-cdaf-428c-8c6d-34d8cd9aa444"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:41:53.697717 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:53.697692    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cdc66edf-cdaf-428c-8c6d-34d8cd9aa444" (UID: "cdc66edf-cdaf-428c-8c6d-34d8cd9aa444"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:41:53.697856 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:53.697840    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-kube-api-access-gcfzp" (OuterVolumeSpecName: "kube-api-access-gcfzp") pod "cdc66edf-cdaf-428c-8c6d-34d8cd9aa444" (UID: "cdc66edf-cdaf-428c-8c6d-34d8cd9aa444"). InnerVolumeSpecName "kube-api-access-gcfzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:41:53.796494 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:53.796453    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcfzp\" (UniqueName: \"kubernetes.io/projected/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-kube-api-access-gcfzp\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:41:53.796494 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:53.796491    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-openshift-service-ca\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:41:53.796494 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:53.796520    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444-maas-api-tls\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:41:54.055659 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:54.055583    2579 generic.go:356] "Generic (PLEG): container finished" podID="cdc66edf-cdaf-428c-8c6d-34d8cd9aa444" containerID="b3a9b1fb4046ae2e607efe93fa398dbd0a6c34400c95d4090097c8368b420543" exitCode=0
Jul 10 02:41:54.055659 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:54.055627    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"
Jul 10 02:41:54.055834 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:54.055669    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x" event={"ID":"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444","Type":"ContainerDied","Data":"b3a9b1fb4046ae2e607efe93fa398dbd0a6c34400c95d4090097c8368b420543"}
Jul 10 02:41:54.055834 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:54.055720    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x" event={"ID":"cdc66edf-cdaf-428c-8c6d-34d8cd9aa444","Type":"ContainerDied","Data":"c14359c033245b92ca314b7b9f440cae7db3968373c436075cdfa022103b113d"}
Jul 10 02:41:54.055834 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:54.055740    2579 scope.go:117] "RemoveContainer" containerID="b3a9b1fb4046ae2e607efe93fa398dbd0a6c34400c95d4090097c8368b420543"
Jul 10 02:41:54.064344 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:54.064327    2579 scope.go:117] "RemoveContainer" containerID="b3a9b1fb4046ae2e607efe93fa398dbd0a6c34400c95d4090097c8368b420543"
Jul 10 02:41:54.064572 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:41:54.064556    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3a9b1fb4046ae2e607efe93fa398dbd0a6c34400c95d4090097c8368b420543\": container with ID starting with b3a9b1fb4046ae2e607efe93fa398dbd0a6c34400c95d4090097c8368b420543 not found: ID does not exist" containerID="b3a9b1fb4046ae2e607efe93fa398dbd0a6c34400c95d4090097c8368b420543"
Jul 10 02:41:54.064619 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:54.064579    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3a9b1fb4046ae2e607efe93fa398dbd0a6c34400c95d4090097c8368b420543"} err="failed to get container status \"b3a9b1fb4046ae2e607efe93fa398dbd0a6c34400c95d4090097c8368b420543\": rpc error: code = NotFound desc = could not find container \"b3a9b1fb4046ae2e607efe93fa398dbd0a6c34400c95d4090097c8368b420543\": container with ID starting with b3a9b1fb4046ae2e607efe93fa398dbd0a6c34400c95d4090097c8368b420543 not found: ID does not exist"
Jul 10 02:41:54.075626 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:54.075600    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"]
Jul 10 02:41:54.079382 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:54.079363    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-57470f4b-5b6fffdc88-k7s2x"]
Jul 10 02:41:55.566625 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:55.566593    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdc66edf-cdaf-428c-8c6d-34d8cd9aa444" path="/var/lib/kubelet/pods/cdc66edf-cdaf-428c-8c6d-34d8cd9aa444/volumes"
Jul 10 02:41:57.758998 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:41:57.758966    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:42:22.868135 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:22.868096    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"]
Jul 10 02:42:22.868597 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:22.868362    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdc66edf-cdaf-428c-8c6d-34d8cd9aa444" containerName="maas-api"
Jul 10 02:42:22.868597 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:22.868373    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdc66edf-cdaf-428c-8c6d-34d8cd9aa444" containerName="maas-api"
Jul 10 02:42:22.868597 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:22.868422    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdc66edf-cdaf-428c-8c6d-34d8cd9aa444" containerName="maas-api"
Jul 10 02:42:22.872323 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:22.872302    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"
Jul 10 02:42:22.875165 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:22.875139    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 02:42:22.875165 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:22.875146    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ad6c59d4-serving-cert\""
Jul 10 02:42:22.876271 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:22.876251    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-scvcw\""
Jul 10 02:42:22.876374 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:22.876341    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 02:42:22.882640 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:22.882621    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"]
Jul 10 02:42:23.029499 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:23.029463    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb8983a9-1ce9-425d-b59c-447aa493814b-maas-api-tls\") pod \"maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8\" (UID: \"bb8983a9-1ce9-425d-b59c-447aa493814b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"
Jul 10 02:42:23.029698 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:23.029540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvh7v\" (UniqueName: \"kubernetes.io/projected/bb8983a9-1ce9-425d-b59c-447aa493814b-kube-api-access-bvh7v\") pod \"maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8\" (UID: \"bb8983a9-1ce9-425d-b59c-447aa493814b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"
Jul 10 02:42:23.029698 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:23.029575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8983a9-1ce9-425d-b59c-447aa493814b-openshift-service-ca\") pod \"maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8\" (UID: \"bb8983a9-1ce9-425d-b59c-447aa493814b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"
Jul 10 02:42:23.130441 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:23.130345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb8983a9-1ce9-425d-b59c-447aa493814b-maas-api-tls\") pod \"maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8\" (UID: \"bb8983a9-1ce9-425d-b59c-447aa493814b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"
Jul 10 02:42:23.130441 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:23.130404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvh7v\" (UniqueName: \"kubernetes.io/projected/bb8983a9-1ce9-425d-b59c-447aa493814b-kube-api-access-bvh7v\") pod \"maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8\" (UID: \"bb8983a9-1ce9-425d-b59c-447aa493814b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"
Jul 10 02:42:23.130671 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:23.130450    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8983a9-1ce9-425d-b59c-447aa493814b-openshift-service-ca\") pod \"maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8\" (UID: \"bb8983a9-1ce9-425d-b59c-447aa493814b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"
Jul 10 02:42:23.131118 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:23.131091    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8983a9-1ce9-425d-b59c-447aa493814b-openshift-service-ca\") pod \"maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8\" (UID: \"bb8983a9-1ce9-425d-b59c-447aa493814b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"
Jul 10 02:42:23.132927 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:23.132909    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb8983a9-1ce9-425d-b59c-447aa493814b-maas-api-tls\") pod \"maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8\" (UID: \"bb8983a9-1ce9-425d-b59c-447aa493814b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"
Jul 10 02:42:23.140630 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:23.140606    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvh7v\" (UniqueName: \"kubernetes.io/projected/bb8983a9-1ce9-425d-b59c-447aa493814b-kube-api-access-bvh7v\") pod \"maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8\" (UID: \"bb8983a9-1ce9-425d-b59c-447aa493814b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"
Jul 10 02:42:23.185052 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:23.185027    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"
Jul 10 02:42:23.313943 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:23.313908    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"]
Jul 10 02:42:23.317003 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:42:23.316968    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb8983a9_1ce9_425d_b59c_447aa493814b.slice/crio-749208ad6ff00be996d254b31589abf8607f058a7b555ef5bb5c6709cf6652e9 WatchSource:0}: Error finding container 749208ad6ff00be996d254b31589abf8607f058a7b555ef5bb5c6709cf6652e9: Status 404 returned error can't find the container with id 749208ad6ff00be996d254b31589abf8607f058a7b555ef5bb5c6709cf6652e9
Jul 10 02:42:24.171945 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:24.171910    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8" event={"ID":"bb8983a9-1ce9-425d-b59c-447aa493814b","Type":"ContainerStarted","Data":"734ff98a76a7d1c30affb0ddf4d34c9899ab55e7f006d55791b055ef0297ba43"}
Jul 10 02:42:24.172399 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:24.171988    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8" event={"ID":"bb8983a9-1ce9-425d-b59c-447aa493814b","Type":"ContainerStarted","Data":"749208ad6ff00be996d254b31589abf8607f058a7b555ef5bb5c6709cf6652e9"}
Jul 10 02:42:24.172399 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:24.172032    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"
Jul 10 02:42:24.189109 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:24.189064    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8" podStartSLOduration=1.8618735050000002 podStartE2EDuration="2.189048805s" totalImagesPullingTime="327.1753ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:42:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:42:24.187331324 +0000 UTC m=+1691.170089009" watchObservedRunningTime="2026-07-10 02:42:24.189048805 +0000 UTC m=+1691.171806493"
Jul 10 02:42:29.553238 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:29.553202    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:42:30.181717 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:30.181691    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"
Jul 10 02:42:37.664617 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.664544    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"]
Jul 10 02:42:37.667927 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.667911    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.670665 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.670638    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ad6c59d4\"/\"openshift-service-ca.crt\""
Jul 10 02:42:37.670810 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.670795    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ad6c59d4\"/\"kube-root-ca.crt\""
Jul 10 02:42:37.671768 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.671751    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ad6c59d4\"/\"default-dockercfg-hrz7z\""
Jul 10 02:42:37.671876 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.671815    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ad6c59d4\"/\"e2e-lifecycle-model-ad6c59d4-kserve-self-signed-certs\""
Jul 10 02:42:37.676943 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.676920    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"]
Jul 10 02:42:37.748293 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.748268    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-model-cache\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.748438 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.748311    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-dshm\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.748438 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.748336    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-tls-certs\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.748438 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.748352    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-tmp-dir\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.748438 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.748397    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-home\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.748438 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.748418    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd69m\" (UniqueName: \"kubernetes.io/projected/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-kube-api-access-bd69m\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.849679 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.849633    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-dshm\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.849794 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.849693    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-tls-certs\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.849794 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.849717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-tmp-dir\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.849794 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.849767    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-home\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.849929 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.849798    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd69m\" (UniqueName: \"kubernetes.io/projected/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-kube-api-access-bd69m\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.849929 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.849835    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-model-cache\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.850202 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.850179    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-home\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.850279 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.850196    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-tmp-dir\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.850279 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.850244    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-model-cache\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.851995 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.851977    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-dshm\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.852280 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.852264    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-tls-certs\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.857149 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.857127    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd69m\" (UniqueName: \"kubernetes.io/projected/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-kube-api-access-bd69m\") pod \"e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") " pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:37.977524 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:37.977429    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:38.107577 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:38.107553    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"]
Jul 10 02:42:38.110184 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:42:38.110155    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e6acdaf_622f_4812_abf1_a3d5d7aa9054.slice/crio-9ce7ef05641a4325b55e292d6269a5f5d258c21a049bb8e4e7fc69e38afb598f WatchSource:0}: Error finding container 9ce7ef05641a4325b55e292d6269a5f5d258c21a049bb8e4e7fc69e38afb598f: Status 404 returned error can't find the container with id 9ce7ef05641a4325b55e292d6269a5f5d258c21a049bb8e4e7fc69e38afb598f
Jul 10 02:42:38.230287 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:38.230198    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn" event={"ID":"5e6acdaf-622f-4812-abf1-a3d5d7aa9054","Type":"ContainerStarted","Data":"5ac10ffee6552d1cba2ba6df21b217dfb8130df33e17b4d22027b94ffeff398f"}
Jul 10 02:42:38.235212 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:38.230236    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn" event={"ID":"5e6acdaf-622f-4812-abf1-a3d5d7aa9054","Type":"ContainerStarted","Data":"9ce7ef05641a4325b55e292d6269a5f5d258c21a049bb8e4e7fc69e38afb598f"}
Jul 10 02:42:38.249991 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:38.249941    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn" podStartSLOduration=1.2499264509999999 podStartE2EDuration="1.249926451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:42:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:42:38.249604791 +0000 UTC m=+1705.232362499" watchObservedRunningTime="2026-07-10 02:42:38.249926451 +0000 UTC m=+1705.232684136"
Jul 10 02:42:41.566345 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:41.566317    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:42:47.978279 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:47.978190    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:47.978279 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:47.978274    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:47.990736 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:47.990702    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:42:48.281686 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:42:48.281610    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:43:02.261541 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:02.261486    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:43:05.191425 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:05.191392    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"]
Jul 10 02:43:05.191924 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:05.191610    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8" podUID="bb8983a9-1ce9-425d-b59c-447aa493814b" containerName="maas-api" containerID="cri-o://734ff98a76a7d1c30affb0ddf4d34c9899ab55e7f006d55791b055ef0297ba43" gracePeriod=30
Jul 10 02:43:05.347206 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:05.347165    2579 generic.go:356] "Generic (PLEG): container finished" podID="bb8983a9-1ce9-425d-b59c-447aa493814b" containerID="734ff98a76a7d1c30affb0ddf4d34c9899ab55e7f006d55791b055ef0297ba43" exitCode=0
Jul 10 02:43:05.347355 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:05.347240    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8" event={"ID":"bb8983a9-1ce9-425d-b59c-447aa493814b","Type":"ContainerDied","Data":"734ff98a76a7d1c30affb0ddf4d34c9899ab55e7f006d55791b055ef0297ba43"}
Jul 10 02:43:05.436088 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:05.436061    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"
Jul 10 02:43:05.575872 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:05.575796    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb8983a9-1ce9-425d-b59c-447aa493814b-maas-api-tls\") pod \"bb8983a9-1ce9-425d-b59c-447aa493814b\" (UID: \"bb8983a9-1ce9-425d-b59c-447aa493814b\") "
Jul 10 02:43:05.575872 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:05.575835    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8983a9-1ce9-425d-b59c-447aa493814b-openshift-service-ca\") pod \"bb8983a9-1ce9-425d-b59c-447aa493814b\" (UID: \"bb8983a9-1ce9-425d-b59c-447aa493814b\") "
Jul 10 02:43:05.576090 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:05.575888    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvh7v\" (UniqueName: \"kubernetes.io/projected/bb8983a9-1ce9-425d-b59c-447aa493814b-kube-api-access-bvh7v\") pod \"bb8983a9-1ce9-425d-b59c-447aa493814b\" (UID: \"bb8983a9-1ce9-425d-b59c-447aa493814b\") "
Jul 10 02:43:05.576258 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:05.576235    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb8983a9-1ce9-425d-b59c-447aa493814b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bb8983a9-1ce9-425d-b59c-447aa493814b" (UID: "bb8983a9-1ce9-425d-b59c-447aa493814b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:43:05.577945 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:05.577920    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb8983a9-1ce9-425d-b59c-447aa493814b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bb8983a9-1ce9-425d-b59c-447aa493814b" (UID: "bb8983a9-1ce9-425d-b59c-447aa493814b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:43:05.578035 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:05.577981    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb8983a9-1ce9-425d-b59c-447aa493814b-kube-api-access-bvh7v" (OuterVolumeSpecName: "kube-api-access-bvh7v") pod "bb8983a9-1ce9-425d-b59c-447aa493814b" (UID: "bb8983a9-1ce9-425d-b59c-447aa493814b"). InnerVolumeSpecName "kube-api-access-bvh7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:43:05.676535 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:05.676486    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb8983a9-1ce9-425d-b59c-447aa493814b-maas-api-tls\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:43:05.676535 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:05.676528    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb8983a9-1ce9-425d-b59c-447aa493814b-openshift-service-ca\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:43:05.676535 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:05.676538    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvh7v\" (UniqueName: \"kubernetes.io/projected/bb8983a9-1ce9-425d-b59c-447aa493814b-kube-api-access-bvh7v\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:43:06.352186 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:06.352153    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"
Jul 10 02:43:06.352636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:06.352161    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8" event={"ID":"bb8983a9-1ce9-425d-b59c-447aa493814b","Type":"ContainerDied","Data":"749208ad6ff00be996d254b31589abf8607f058a7b555ef5bb5c6709cf6652e9"}
Jul 10 02:43:06.352636 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:06.352281    2579 scope.go:117] "RemoveContainer" containerID="734ff98a76a7d1c30affb0ddf4d34c9899ab55e7f006d55791b055ef0297ba43"
Jul 10 02:43:06.372193 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:06.372165    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"]
Jul 10 02:43:06.374429 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:06.374408    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ad6c59d4-775df7b555-zcvm8"]
Jul 10 02:43:07.565610 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:07.565556    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb8983a9-1ce9-425d-b59c-447aa493814b" path="/var/lib/kubelet/pods/bb8983a9-1ce9-425d-b59c-447aa493814b/volumes"
Jul 10 02:43:15.958606 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:15.958564    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"]
Jul 10 02:43:15.969519 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:15.969479    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:43:30.978811 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:30.978767    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn" podUID="5e6acdaf-622f-4812-abf1-a3d5d7aa9054" containerName="main" containerID="cri-o://5ac10ffee6552d1cba2ba6df21b217dfb8130df33e17b4d22027b94ffeff398f" gracePeriod=45
Jul 10 02:43:31.232037 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.231977    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:43:31.394129 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.394081    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-dshm\") pod \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") "
Jul 10 02:43:31.394324 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.394163    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bd69m\" (UniqueName: \"kubernetes.io/projected/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-kube-api-access-bd69m\") pod \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") "
Jul 10 02:43:31.394324 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.394182    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-tmp-dir\") pod \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") "
Jul 10 02:43:31.394324 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.394213    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-home\") pod \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") "
Jul 10 02:43:31.394324 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.394322    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-model-cache\") pod \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") "
Jul 10 02:43:31.394579 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.394360    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-tls-certs\") pod \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\" (UID: \"5e6acdaf-622f-4812-abf1-a3d5d7aa9054\") "
Jul 10 02:43:31.394579 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.394456    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5e6acdaf-622f-4812-abf1-a3d5d7aa9054" (UID: "5e6acdaf-622f-4812-abf1-a3d5d7aa9054"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:43:31.394579 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.394466    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-home" (OuterVolumeSpecName: "home") pod "5e6acdaf-622f-4812-abf1-a3d5d7aa9054" (UID: "5e6acdaf-622f-4812-abf1-a3d5d7aa9054"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:43:31.394579 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.394571    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-model-cache" (OuterVolumeSpecName: "model-cache") pod "5e6acdaf-622f-4812-abf1-a3d5d7aa9054" (UID: "5e6acdaf-622f-4812-abf1-a3d5d7aa9054"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:43:31.394776 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.394685    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-home\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:43:31.394776 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.394698    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-model-cache\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:43:31.394776 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.394707    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-tmp-dir\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:43:31.396544 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.396491    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-kube-api-access-bd69m" (OuterVolumeSpecName: "kube-api-access-bd69m") pod "5e6acdaf-622f-4812-abf1-a3d5d7aa9054" (UID: "5e6acdaf-622f-4812-abf1-a3d5d7aa9054"). InnerVolumeSpecName "kube-api-access-bd69m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:43:31.396544 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.396526    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-dshm" (OuterVolumeSpecName: "dshm") pod "5e6acdaf-622f-4812-abf1-a3d5d7aa9054" (UID: "5e6acdaf-622f-4812-abf1-a3d5d7aa9054"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:43:31.396680 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.396586    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5e6acdaf-622f-4812-abf1-a3d5d7aa9054" (UID: "5e6acdaf-622f-4812-abf1-a3d5d7aa9054"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:43:31.450668 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.450636    2579 generic.go:356] "Generic (PLEG): container finished" podID="5e6acdaf-622f-4812-abf1-a3d5d7aa9054" containerID="5ac10ffee6552d1cba2ba6df21b217dfb8130df33e17b4d22027b94ffeff398f" exitCode=0
Jul 10 02:43:31.450843 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.450689    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"
Jul 10 02:43:31.450843 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.450717    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn" event={"ID":"5e6acdaf-622f-4812-abf1-a3d5d7aa9054","Type":"ContainerDied","Data":"5ac10ffee6552d1cba2ba6df21b217dfb8130df33e17b4d22027b94ffeff398f"}
Jul 10 02:43:31.450843 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.450759    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn" event={"ID":"5e6acdaf-622f-4812-abf1-a3d5d7aa9054","Type":"ContainerDied","Data":"9ce7ef05641a4325b55e292d6269a5f5d258c21a049bb8e4e7fc69e38afb598f"}
Jul 10 02:43:31.450843 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.450775    2579 scope.go:117] "RemoveContainer" containerID="5ac10ffee6552d1cba2ba6df21b217dfb8130df33e17b4d22027b94ffeff398f"
Jul 10 02:43:31.459211 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.459192    2579 scope.go:117] "RemoveContainer" containerID="5ac10ffee6552d1cba2ba6df21b217dfb8130df33e17b4d22027b94ffeff398f"
Jul 10 02:43:31.459469 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:43:31.459449    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ac10ffee6552d1cba2ba6df21b217dfb8130df33e17b4d22027b94ffeff398f\": container with ID starting with 5ac10ffee6552d1cba2ba6df21b217dfb8130df33e17b4d22027b94ffeff398f not found: ID does not exist" containerID="5ac10ffee6552d1cba2ba6df21b217dfb8130df33e17b4d22027b94ffeff398f"
Jul 10 02:43:31.459571 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.459476    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ac10ffee6552d1cba2ba6df21b217dfb8130df33e17b4d22027b94ffeff398f"} err="failed to get container status \"5ac10ffee6552d1cba2ba6df21b217dfb8130df33e17b4d22027b94ffeff398f\": rpc error: code = NotFound desc = could not find container \"5ac10ffee6552d1cba2ba6df21b217dfb8130df33e17b4d22027b94ffeff398f\": container with ID starting with 5ac10ffee6552d1cba2ba6df21b217dfb8130df33e17b4d22027b94ffeff398f not found: ID does not exist"
Jul 10 02:43:31.471076 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.471048    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"]
Jul 10 02:43:31.476906 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.476885    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ad6c59d4/e2e-lifecycle-model-ad6c59d4-kserve-d5fcc7755-72jhn"]
Jul 10 02:43:31.495108 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.495050    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-dshm\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:43:31.495108 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.495071    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd69m\" (UniqueName: \"kubernetes.io/projected/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-kube-api-access-bd69m\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:43:31.495108 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.495081    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6acdaf-622f-4812-abf1-a3d5d7aa9054-tls-certs\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:43:31.566104 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:31.566077    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e6acdaf-622f-4812-abf1-a3d5d7aa9054" path="/var/lib/kubelet/pods/5e6acdaf-622f-4812-abf1-a3d5d7aa9054/volumes"
Jul 10 02:43:55.457812 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:43:55.457775    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:44:12.259899 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:44:12.259865    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:44:26.765707 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:44:26.765671    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:45:06.662011 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.661979    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"]
Jul 10 02:45:06.662432 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.662282    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb8983a9-1ce9-425d-b59c-447aa493814b" containerName="maas-api"
Jul 10 02:45:06.662432 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.662294    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb8983a9-1ce9-425d-b59c-447aa493814b" containerName="maas-api"
Jul 10 02:45:06.662432 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.662311    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e6acdaf-622f-4812-abf1-a3d5d7aa9054" containerName="main"
Jul 10 02:45:06.662432 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.662317    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e6acdaf-622f-4812-abf1-a3d5d7aa9054" containerName="main"
Jul 10 02:45:06.662432 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.662364    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb8983a9-1ce9-425d-b59c-447aa493814b" containerName="maas-api"
Jul 10 02:45:06.662432 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.662372    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e6acdaf-622f-4812-abf1-a3d5d7aa9054" containerName="main"
Jul 10 02:45:06.665155 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.665138    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:45:06.667848 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.667819    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-2f242d-serving-cert\""
Jul 10 02:45:06.667978 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.667834    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 02:45:06.668294 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.668094    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 02:45:06.668912 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.668886    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-scvcw\""
Jul 10 02:45:06.676993 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.676966    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"]
Jul 10 02:45:06.713316 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.713279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/594104f2-dc53-4d7d-afde-cf888545bd4b-maas-api-tls\") pod \"maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l\" (UID: \"594104f2-dc53-4d7d-afde-cf888545bd4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:45:06.713472 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.713323    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/594104f2-dc53-4d7d-afde-cf888545bd4b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l\" (UID: \"594104f2-dc53-4d7d-afde-cf888545bd4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:45:06.713472 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.713401    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwcst\" (UniqueName: \"kubernetes.io/projected/594104f2-dc53-4d7d-afde-cf888545bd4b-kube-api-access-hwcst\") pod \"maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l\" (UID: \"594104f2-dc53-4d7d-afde-cf888545bd4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:45:06.813831 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.813784    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/594104f2-dc53-4d7d-afde-cf888545bd4b-maas-api-tls\") pod \"maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l\" (UID: \"594104f2-dc53-4d7d-afde-cf888545bd4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:45:06.813831 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.813831    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/594104f2-dc53-4d7d-afde-cf888545bd4b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l\" (UID: \"594104f2-dc53-4d7d-afde-cf888545bd4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:45:06.814109 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.813864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwcst\" (UniqueName: \"kubernetes.io/projected/594104f2-dc53-4d7d-afde-cf888545bd4b-kube-api-access-hwcst\") pod \"maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l\" (UID: \"594104f2-dc53-4d7d-afde-cf888545bd4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:45:06.814109 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:45:06.813936    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-serving-cert: secret "maas-api-e2e-inf-a-2f242d-serving-cert" not found
Jul 10 02:45:06.814109 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:45:06.814011    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/594104f2-dc53-4d7d-afde-cf888545bd4b-maas-api-tls podName:594104f2-dc53-4d7d-afde-cf888545bd4b nodeName:}" failed. No retries permitted until 2026-07-10 02:45:07.313994114 +0000 UTC m=+1854.296751778 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/594104f2-dc53-4d7d-afde-cf888545bd4b-maas-api-tls") pod "maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l" (UID: "594104f2-dc53-4d7d-afde-cf888545bd4b") : secret "maas-api-e2e-inf-a-2f242d-serving-cert" not found
Jul 10 02:45:06.814606 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.814584    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/594104f2-dc53-4d7d-afde-cf888545bd4b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l\" (UID: \"594104f2-dc53-4d7d-afde-cf888545bd4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:45:06.823106 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:06.823086    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwcst\" (UniqueName: \"kubernetes.io/projected/594104f2-dc53-4d7d-afde-cf888545bd4b-kube-api-access-hwcst\") pod \"maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l\" (UID: \"594104f2-dc53-4d7d-afde-cf888545bd4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:45:07.319839 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:07.319802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/594104f2-dc53-4d7d-afde-cf888545bd4b-maas-api-tls\") pod \"maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l\" (UID: \"594104f2-dc53-4d7d-afde-cf888545bd4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:45:07.322298 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:07.322275    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/594104f2-dc53-4d7d-afde-cf888545bd4b-maas-api-tls\") pod \"maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l\" (UID: \"594104f2-dc53-4d7d-afde-cf888545bd4b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:45:07.577891 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:07.577806    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:45:07.706148 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:07.706121    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"]
Jul 10 02:45:07.708477 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:45:07.708449    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod594104f2_dc53_4d7d_afde_cf888545bd4b.slice/crio-403734eed2a6d3be83f0cf5dbbfa2b6db81d3a0f4c382b6e4c4f7b498091d674 WatchSource:0}: Error finding container 403734eed2a6d3be83f0cf5dbbfa2b6db81d3a0f4c382b6e4c4f7b498091d674: Status 404 returned error can't find the container with id 403734eed2a6d3be83f0cf5dbbfa2b6db81d3a0f4c382b6e4c4f7b498091d674
Jul 10 02:45:07.709774 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:07.709760    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:45:07.831206 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:07.831126    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l" event={"ID":"594104f2-dc53-4d7d-afde-cf888545bd4b","Type":"ContainerStarted","Data":"403734eed2a6d3be83f0cf5dbbfa2b6db81d3a0f4c382b6e4c4f7b498091d674"}
Jul 10 02:45:08.837220 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:08.837189    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l" event={"ID":"594104f2-dc53-4d7d-afde-cf888545bd4b","Type":"ContainerStarted","Data":"6148334edd3d9454d761572850bca72a882a795a16029d889c28810c864a48b2"}
Jul 10 02:45:08.837653 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:08.837313    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:45:08.854733 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:08.854672    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l" podStartSLOduration=2.337347457 podStartE2EDuration="2.854653678s" totalImagesPullingTime="517.306221ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:08.854025763 +0000 UTC m=+1855.836783450" watchObservedRunningTime="2026-07-10 02:45:08.854653678 +0000 UTC m=+1855.837411368"
Jul 10 02:45:14.692371 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:14.692336    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"]
Jul 10 02:45:14.696217 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:14.696193    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"
Jul 10 02:45:14.699035 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:14.699013    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-1238eb-serving-cert\""
Jul 10 02:45:14.705562 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:14.705542    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"]
Jul 10 02:45:14.778008 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:14.777974    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57e22f70-58f7-4b66-abd6-fb946a667995-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms\" (UID: \"57e22f70-58f7-4b66-abd6-fb946a667995\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"
Jul 10 02:45:14.778171 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:14.778042    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjph7\" (UniqueName: \"kubernetes.io/projected/57e22f70-58f7-4b66-abd6-fb946a667995-kube-api-access-vjph7\") pod \"maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms\" (UID: \"57e22f70-58f7-4b66-abd6-fb946a667995\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"
Jul 10 02:45:14.778171 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:14.778070    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57e22f70-58f7-4b66-abd6-fb946a667995-maas-api-tls\") pod \"maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms\" (UID: \"57e22f70-58f7-4b66-abd6-fb946a667995\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"
Jul 10 02:45:14.847421 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:14.847390    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:45:14.879262 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:14.879235    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57e22f70-58f7-4b66-abd6-fb946a667995-maas-api-tls\") pod \"maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms\" (UID: \"57e22f70-58f7-4b66-abd6-fb946a667995\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"
Jul 10 02:45:14.879434 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:14.879290    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57e22f70-58f7-4b66-abd6-fb946a667995-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms\" (UID: \"57e22f70-58f7-4b66-abd6-fb946a667995\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"
Jul 10 02:45:14.879434 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:14.879367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjph7\" (UniqueName: \"kubernetes.io/projected/57e22f70-58f7-4b66-abd6-fb946a667995-kube-api-access-vjph7\") pod \"maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms\" (UID: \"57e22f70-58f7-4b66-abd6-fb946a667995\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"
Jul 10 02:45:14.880462 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:14.880434    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57e22f70-58f7-4b66-abd6-fb946a667995-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms\" (UID: \"57e22f70-58f7-4b66-abd6-fb946a667995\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"
Jul 10 02:45:14.881838 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:14.881814    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57e22f70-58f7-4b66-abd6-fb946a667995-maas-api-tls\") pod \"maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms\" (UID: \"57e22f70-58f7-4b66-abd6-fb946a667995\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"
Jul 10 02:45:14.890906 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:14.890881    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjph7\" (UniqueName: \"kubernetes.io/projected/57e22f70-58f7-4b66-abd6-fb946a667995-kube-api-access-vjph7\") pod \"maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms\" (UID: \"57e22f70-58f7-4b66-abd6-fb946a667995\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"
Jul 10 02:45:15.008002 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:15.007916    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"
Jul 10 02:45:15.077991 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:15.077959    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:45:15.138657 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:15.138633    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"]
Jul 10 02:45:15.141092 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:45:15.141067    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e22f70_58f7_4b66_abd6_fb946a667995.slice/crio-209933d79d7d192c936ca7b794f279691ccf09941d41a41fd63035383ce97902 WatchSource:0}: Error finding container 209933d79d7d192c936ca7b794f279691ccf09941d41a41fd63035383ce97902: Status 404 returned error can't find the container with id 209933d79d7d192c936ca7b794f279691ccf09941d41a41fd63035383ce97902
Jul 10 02:45:15.865257 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:15.865226    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms" event={"ID":"57e22f70-58f7-4b66-abd6-fb946a667995","Type":"ContainerStarted","Data":"04bfdd9e9aafd2709d657b59c915273e75f6e76af631f23c55e40fdd0dd60901"}
Jul 10 02:45:15.865690 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:15.865274    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms" event={"ID":"57e22f70-58f7-4b66-abd6-fb946a667995","Type":"ContainerStarted","Data":"209933d79d7d192c936ca7b794f279691ccf09941d41a41fd63035383ce97902"}
Jul 10 02:45:15.866309 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:15.866286    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"
Jul 10 02:45:15.887144 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:15.887087    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms" podStartSLOduration=1.497839463 podStartE2EDuration="1.88706973s" totalImagesPullingTime="389.230267ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:15.884832984 +0000 UTC m=+1862.867590669" watchObservedRunningTime="2026-07-10 02:45:15.88706973 +0000 UTC m=+1862.869827417"
Jul 10 02:45:22.879216 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:22.879188    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"
Jul 10 02:45:26.263737 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:26.263703    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:45:44.967441 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:44.967363    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"]
Jul 10 02:45:44.971301 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:44.971284    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:44.973825 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:44.973800    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-2f242d\"/\"openshift-service-ca.crt\""
Jul 10 02:45:44.975084 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:44.975063    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-2f242d\"/\"test-model-2f242d-kserve-self-signed-certs\""
Jul 10 02:45:44.975193 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:44.975068    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-2f242d\"/\"default-dockercfg-hmp2k\""
Jul 10 02:45:44.975193 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:44.975101    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-2f242d\"/\"kube-root-ca.crt\""
Jul 10 02:45:44.980638 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:44.980616    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"]
Jul 10 02:45:45.128043 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.128007    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84hsv\" (UniqueName: \"kubernetes.io/projected/dca0d150-61c7-48cd-8186-928c6e52b437-kube-api-access-84hsv\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.128198 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.128061    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-model-cache\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.128198 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.128131    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-tmp-dir\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.128198 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.128165    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dca0d150-61c7-48cd-8186-928c6e52b437-tls-certs\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.128198 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.128188    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-dshm\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.128342 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.128226    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-home\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.229007 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.228917    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84hsv\" (UniqueName: \"kubernetes.io/projected/dca0d150-61c7-48cd-8186-928c6e52b437-kube-api-access-84hsv\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.229007 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.228981    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-model-cache\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.229194 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.229029    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-tmp-dir\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.229194 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.229060    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dca0d150-61c7-48cd-8186-928c6e52b437-tls-certs\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.229194 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.229084    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-dshm\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.229194 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.229129    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-home\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.229622 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.229592    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-home\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.229729 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.229617    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-tmp-dir\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.229729 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.229701    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-model-cache\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.231515 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.231482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-dshm\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.231817 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.231802    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dca0d150-61c7-48cd-8186-928c6e52b437-tls-certs\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.236147 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.236123    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84hsv\" (UniqueName: \"kubernetes.io/projected/dca0d150-61c7-48cd-8186-928c6e52b437-kube-api-access-84hsv\") pod \"test-model-2f242d-kserve-6586f86fb9-j2jnz\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") " pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.281269 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.281234    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:45.406614 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.406591    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"]
Jul 10 02:45:45.409305 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:45:45.409281    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddca0d150_61c7_48cd_8186_928c6e52b437.slice/crio-542b87fb4b197161458c48fd95faf200d5cc16d1d937bbce48da8f938ebb48d9 WatchSource:0}: Error finding container 542b87fb4b197161458c48fd95faf200d5cc16d1d937bbce48da8f938ebb48d9: Status 404 returned error can't find the container with id 542b87fb4b197161458c48fd95faf200d5cc16d1d937bbce48da8f938ebb48d9
Jul 10 02:45:45.987270 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.987235    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz" event={"ID":"dca0d150-61c7-48cd-8186-928c6e52b437","Type":"ContainerStarted","Data":"a830bdc5f02b577746a03333c8dfcdc5bc3721389af4d9a1702c85a94dfbe989"}
Jul 10 02:45:45.987270 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:45.987273    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz" event={"ID":"dca0d150-61c7-48cd-8186-928c6e52b437","Type":"ContainerStarted","Data":"542b87fb4b197161458c48fd95faf200d5cc16d1d937bbce48da8f938ebb48d9"}
Jul 10 02:45:46.007216 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:46.007166    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz" podStartSLOduration=2.007148691 podStartE2EDuration="2.007148691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:46.006699466 +0000 UTC m=+1892.989457165" watchObservedRunningTime="2026-07-10 02:45:46.007148691 +0000 UTC m=+1892.989906379"
Jul 10 02:45:48.365225 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:48.365189    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:45:55.281514 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:55.281479    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:55.281963 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:55.281538    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:55.294012 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:55.293991    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:45:56.043402 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:45:56.043369    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:46:10.458661 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.458629    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"]
Jul 10 02:46:10.461964 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.461944    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.464763 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.464737    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1238eb\"/\"kube-root-ca.crt\""
Jul 10 02:46:10.464886 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.464799    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1238eb\"/\"openshift-service-ca.crt\""
Jul 10 02:46:10.465996 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.465975    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1238eb\"/\"test-model-1238eb-kserve-self-signed-certs\""
Jul 10 02:46:10.466142 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.466001    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1238eb\"/\"default-dockercfg-f97fz\""
Jul 10 02:46:10.473148 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.473125    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"]
Jul 10 02:46:10.541071 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.541039    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-dshm\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.541071 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.541074    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-tmp-dir\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.541269 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.541106    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqkzj\" (UniqueName: \"kubernetes.io/projected/29910a03-656c-47dc-ba30-24b9923b233a-kube-api-access-xqkzj\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.541269 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.541172    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-home\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.541269 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.541220    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29910a03-656c-47dc-ba30-24b9923b233a-tls-certs\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.541367 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.541278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-model-cache\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.641615 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.641584    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-model-cache\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.641770 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.641627    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-dshm\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.641770 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.641644    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-tmp-dir\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.641836 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.641778    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqkzj\" (UniqueName: \"kubernetes.io/projected/29910a03-656c-47dc-ba30-24b9923b233a-kube-api-access-xqkzj\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.641836 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.641826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-home\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.641904 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.641865    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29910a03-656c-47dc-ba30-24b9923b233a-tls-certs\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.642059 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.642040    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-tmp-dir\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.642096 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.642072    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-model-cache\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.642210 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.642194    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-home\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.644133 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.644110    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-dshm\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.644243 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.644228    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29910a03-656c-47dc-ba30-24b9923b233a-tls-certs\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.650010 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.649988    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqkzj\" (UniqueName: \"kubernetes.io/projected/29910a03-656c-47dc-ba30-24b9923b233a-kube-api-access-xqkzj\") pod \"test-model-1238eb-kserve-79f88955f-mrqt7\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") " pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.772629 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.772547    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:10.898827 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:10.898803    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"]
Jul 10 02:46:10.901153 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:46:10.901130    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29910a03_656c_47dc_ba30_24b9923b233a.slice/crio-cb61032bf8213c70fe04359a0409631e2bb66d9e5588fba068e0b6c6cb70d1bc WatchSource:0}: Error finding container cb61032bf8213c70fe04359a0409631e2bb66d9e5588fba068e0b6c6cb70d1bc: Status 404 returned error can't find the container with id cb61032bf8213c70fe04359a0409631e2bb66d9e5588fba068e0b6c6cb70d1bc
Jul 10 02:46:11.092202 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:11.092116    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7" event={"ID":"29910a03-656c-47dc-ba30-24b9923b233a","Type":"ContainerStarted","Data":"f50d706a3c78b6b1e59d00451d52e3df3edd079322822f2f28e14bc790e41a7d"}
Jul 10 02:46:11.092202 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:11.092158    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7" event={"ID":"29910a03-656c-47dc-ba30-24b9923b233a","Type":"ContainerStarted","Data":"cb61032bf8213c70fe04359a0409631e2bb66d9e5588fba068e0b6c6cb70d1bc"}
Jul 10 02:46:11.110000 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:11.109950    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7" podStartSLOduration=1.109934148 podStartE2EDuration="1.109934148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:46:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:46:11.108986773 +0000 UTC m=+1918.091744460" watchObservedRunningTime="2026-07-10 02:46:11.109934148 +0000 UTC m=+1918.092691834"
Jul 10 02:46:16.156941 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:16.156909    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:46:20.773288 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:20.773250    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:20.773739 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:20.773328    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:20.786399 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:20.786379    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:21.146732 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:21.146704    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:28.445207 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:28.445175    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"]
Jul 10 02:46:29.366993 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:29.366958    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"]
Jul 10 02:46:30.220978 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:30.220939    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"]
Jul 10 02:46:30.221432 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:30.221230    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l" podUID="594104f2-dc53-4d7d-afde-cf888545bd4b" containerName="maas-api" containerID="cri-o://6148334edd3d9454d761572850bca72a882a795a16029d889c28810c864a48b2" gracePeriod=30
Jul 10 02:46:30.459145 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:30.459117    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:46:30.606369 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:30.606344    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/594104f2-dc53-4d7d-afde-cf888545bd4b-maas-api-tls\") pod \"594104f2-dc53-4d7d-afde-cf888545bd4b\" (UID: \"594104f2-dc53-4d7d-afde-cf888545bd4b\") "
Jul 10 02:46:30.606536 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:30.606394    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/594104f2-dc53-4d7d-afde-cf888545bd4b-openshift-service-ca\") pod \"594104f2-dc53-4d7d-afde-cf888545bd4b\" (UID: \"594104f2-dc53-4d7d-afde-cf888545bd4b\") "
Jul 10 02:46:30.606536 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:30.606423    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwcst\" (UniqueName: \"kubernetes.io/projected/594104f2-dc53-4d7d-afde-cf888545bd4b-kube-api-access-hwcst\") pod \"594104f2-dc53-4d7d-afde-cf888545bd4b\" (UID: \"594104f2-dc53-4d7d-afde-cf888545bd4b\") "
Jul 10 02:46:30.606773 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:30.606751    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/594104f2-dc53-4d7d-afde-cf888545bd4b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "594104f2-dc53-4d7d-afde-cf888545bd4b" (UID: "594104f2-dc53-4d7d-afde-cf888545bd4b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:46:30.608635 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:30.608613    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/594104f2-dc53-4d7d-afde-cf888545bd4b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "594104f2-dc53-4d7d-afde-cf888545bd4b" (UID: "594104f2-dc53-4d7d-afde-cf888545bd4b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:46:30.608778 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:30.608754    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/594104f2-dc53-4d7d-afde-cf888545bd4b-kube-api-access-hwcst" (OuterVolumeSpecName: "kube-api-access-hwcst") pod "594104f2-dc53-4d7d-afde-cf888545bd4b" (UID: "594104f2-dc53-4d7d-afde-cf888545bd4b"). InnerVolumeSpecName "kube-api-access-hwcst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:46:30.707741 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:30.707708    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/594104f2-dc53-4d7d-afde-cf888545bd4b-openshift-service-ca\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:46:30.707741 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:30.707735    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwcst\" (UniqueName: \"kubernetes.io/projected/594104f2-dc53-4d7d-afde-cf888545bd4b-kube-api-access-hwcst\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:46:30.707741 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:30.707746    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/594104f2-dc53-4d7d-afde-cf888545bd4b-maas-api-tls\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:46:31.175932 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:31.175899    2579 generic.go:356] "Generic (PLEG): container finished" podID="594104f2-dc53-4d7d-afde-cf888545bd4b" containerID="6148334edd3d9454d761572850bca72a882a795a16029d889c28810c864a48b2" exitCode=0
Jul 10 02:46:31.176115 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:31.175940    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"
Jul 10 02:46:31.176115 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:31.175979    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l" event={"ID":"594104f2-dc53-4d7d-afde-cf888545bd4b","Type":"ContainerDied","Data":"6148334edd3d9454d761572850bca72a882a795a16029d889c28810c864a48b2"}
Jul 10 02:46:31.176115 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:31.176030    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l" event={"ID":"594104f2-dc53-4d7d-afde-cf888545bd4b","Type":"ContainerDied","Data":"403734eed2a6d3be83f0cf5dbbfa2b6db81d3a0f4c382b6e4c4f7b498091d674"}
Jul 10 02:46:31.176115 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:31.176059    2579 scope.go:117] "RemoveContainer" containerID="6148334edd3d9454d761572850bca72a882a795a16029d889c28810c864a48b2"
Jul 10 02:46:31.185025 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:31.185009    2579 scope.go:117] "RemoveContainer" containerID="6148334edd3d9454d761572850bca72a882a795a16029d889c28810c864a48b2"
Jul 10 02:46:31.185265 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:46:31.185242    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6148334edd3d9454d761572850bca72a882a795a16029d889c28810c864a48b2\": container with ID starting with 6148334edd3d9454d761572850bca72a882a795a16029d889c28810c864a48b2 not found: ID does not exist" containerID="6148334edd3d9454d761572850bca72a882a795a16029d889c28810c864a48b2"
Jul 10 02:46:31.185326 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:31.185272    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6148334edd3d9454d761572850bca72a882a795a16029d889c28810c864a48b2"} err="failed to get container status \"6148334edd3d9454d761572850bca72a882a795a16029d889c28810c864a48b2\": rpc error: code = NotFound desc = could not find container \"6148334edd3d9454d761572850bca72a882a795a16029d889c28810c864a48b2\": container with ID starting with 6148334edd3d9454d761572850bca72a882a795a16029d889c28810c864a48b2 not found: ID does not exist"
Jul 10 02:46:31.195872 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:31.195854    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"]
Jul 10 02:46:31.199833 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:31.199805    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2f242d-7869c454b5-hqq6l"]
Jul 10 02:46:31.572090 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:31.572003    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="594104f2-dc53-4d7d-afde-cf888545bd4b" path="/var/lib/kubelet/pods/594104f2-dc53-4d7d-afde-cf888545bd4b/volumes"
Jul 10 02:46:36.455961 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:36.455925    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:46:43.465781 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.465719    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz" podUID="dca0d150-61c7-48cd-8186-928c6e52b437" containerName="main" containerID="cri-o://a830bdc5f02b577746a03333c8dfcdc5bc3721389af4d9a1702c85a94dfbe989" gracePeriod=45
Jul 10 02:46:43.713738 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.713712    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:46:43.822190 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.822118    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dca0d150-61c7-48cd-8186-928c6e52b437-tls-certs\") pod \"dca0d150-61c7-48cd-8186-928c6e52b437\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") "
Jul 10 02:46:43.822190 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.822157    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84hsv\" (UniqueName: \"kubernetes.io/projected/dca0d150-61c7-48cd-8186-928c6e52b437-kube-api-access-84hsv\") pod \"dca0d150-61c7-48cd-8186-928c6e52b437\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") "
Jul 10 02:46:43.822395 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.822200    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-model-cache\") pod \"dca0d150-61c7-48cd-8186-928c6e52b437\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") "
Jul 10 02:46:43.822395 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.822233    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-tmp-dir\") pod \"dca0d150-61c7-48cd-8186-928c6e52b437\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") "
Jul 10 02:46:43.822395 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.822278    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-home\") pod \"dca0d150-61c7-48cd-8186-928c6e52b437\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") "
Jul 10 02:46:43.822395 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.822296    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-dshm\") pod \"dca0d150-61c7-48cd-8186-928c6e52b437\" (UID: \"dca0d150-61c7-48cd-8186-928c6e52b437\") "
Jul 10 02:46:43.822622 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.822532    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-model-cache" (OuterVolumeSpecName: "model-cache") pod "dca0d150-61c7-48cd-8186-928c6e52b437" (UID: "dca0d150-61c7-48cd-8186-928c6e52b437"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:46:43.822622 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.822552    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-home" (OuterVolumeSpecName: "home") pod "dca0d150-61c7-48cd-8186-928c6e52b437" (UID: "dca0d150-61c7-48cd-8186-928c6e52b437"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:46:43.822622 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.822565    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dca0d150-61c7-48cd-8186-928c6e52b437" (UID: "dca0d150-61c7-48cd-8186-928c6e52b437"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:46:43.824573 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.824553    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dca0d150-61c7-48cd-8186-928c6e52b437-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dca0d150-61c7-48cd-8186-928c6e52b437" (UID: "dca0d150-61c7-48cd-8186-928c6e52b437"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:46:43.824672 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.824627    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-dshm" (OuterVolumeSpecName: "dshm") pod "dca0d150-61c7-48cd-8186-928c6e52b437" (UID: "dca0d150-61c7-48cd-8186-928c6e52b437"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:46:43.824672 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.824643    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dca0d150-61c7-48cd-8186-928c6e52b437-kube-api-access-84hsv" (OuterVolumeSpecName: "kube-api-access-84hsv") pod "dca0d150-61c7-48cd-8186-928c6e52b437" (UID: "dca0d150-61c7-48cd-8186-928c6e52b437"). InnerVolumeSpecName "kube-api-access-84hsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:46:43.922907 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.922871    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-tmp-dir\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:46:43.922907 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.922898    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-home\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:46:43.922907 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.922907    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-dshm\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:46:43.922907 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.922914    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dca0d150-61c7-48cd-8186-928c6e52b437-tls-certs\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:46:43.923235 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.922924    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84hsv\" (UniqueName: \"kubernetes.io/projected/dca0d150-61c7-48cd-8186-928c6e52b437-kube-api-access-84hsv\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:46:43.923235 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:43.922932    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dca0d150-61c7-48cd-8186-928c6e52b437-model-cache\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:46:44.229479 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.229443    2579 generic.go:356] "Generic (PLEG): container finished" podID="dca0d150-61c7-48cd-8186-928c6e52b437" containerID="a830bdc5f02b577746a03333c8dfcdc5bc3721389af4d9a1702c85a94dfbe989" exitCode=0
Jul 10 02:46:44.229673 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.229486    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"
Jul 10 02:46:44.229673 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.229496    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz" event={"ID":"dca0d150-61c7-48cd-8186-928c6e52b437","Type":"ContainerDied","Data":"a830bdc5f02b577746a03333c8dfcdc5bc3721389af4d9a1702c85a94dfbe989"}
Jul 10 02:46:44.229673 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.229548    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz" event={"ID":"dca0d150-61c7-48cd-8186-928c6e52b437","Type":"ContainerDied","Data":"542b87fb4b197161458c48fd95faf200d5cc16d1d937bbce48da8f938ebb48d9"}
Jul 10 02:46:44.229673 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.229564    2579 scope.go:117] "RemoveContainer" containerID="a830bdc5f02b577746a03333c8dfcdc5bc3721389af4d9a1702c85a94dfbe989"
Jul 10 02:46:44.238757 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.238739    2579 scope.go:117] "RemoveContainer" containerID="a830bdc5f02b577746a03333c8dfcdc5bc3721389af4d9a1702c85a94dfbe989"
Jul 10 02:46:44.238976 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:46:44.238956    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a830bdc5f02b577746a03333c8dfcdc5bc3721389af4d9a1702c85a94dfbe989\": container with ID starting with a830bdc5f02b577746a03333c8dfcdc5bc3721389af4d9a1702c85a94dfbe989 not found: ID does not exist" containerID="a830bdc5f02b577746a03333c8dfcdc5bc3721389af4d9a1702c85a94dfbe989"
Jul 10 02:46:44.239021 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.238985    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a830bdc5f02b577746a03333c8dfcdc5bc3721389af4d9a1702c85a94dfbe989"} err="failed to get container status \"a830bdc5f02b577746a03333c8dfcdc5bc3721389af4d9a1702c85a94dfbe989\": rpc error: code = NotFound desc = could not find container \"a830bdc5f02b577746a03333c8dfcdc5bc3721389af4d9a1702c85a94dfbe989\": container with ID starting with a830bdc5f02b577746a03333c8dfcdc5bc3721389af4d9a1702c85a94dfbe989 not found: ID does not exist"
Jul 10 02:46:44.249383 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.249362    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"]
Jul 10 02:46:44.252859 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.252838    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-2f242d/test-model-2f242d-kserve-6586f86fb9-j2jnz"]
Jul 10 02:46:44.386975 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.386939    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7" podUID="29910a03-656c-47dc-ba30-24b9923b233a" containerName="main" containerID="cri-o://f50d706a3c78b6b1e59d00451d52e3df3edd079322822f2f28e14bc790e41a7d" gracePeriod=45
Jul 10 02:46:44.635147 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.635123    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:44.731132 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.731101    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-dshm\") pod \"29910a03-656c-47dc-ba30-24b9923b233a\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") "
Jul 10 02:46:44.731132 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.731134    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqkzj\" (UniqueName: \"kubernetes.io/projected/29910a03-656c-47dc-ba30-24b9923b233a-kube-api-access-xqkzj\") pod \"29910a03-656c-47dc-ba30-24b9923b233a\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") "
Jul 10 02:46:44.731354 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.731161    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-home\") pod \"29910a03-656c-47dc-ba30-24b9923b233a\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") "
Jul 10 02:46:44.731354 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.731175    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-model-cache\") pod \"29910a03-656c-47dc-ba30-24b9923b233a\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") "
Jul 10 02:46:44.731354 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.731197    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29910a03-656c-47dc-ba30-24b9923b233a-tls-certs\") pod \"29910a03-656c-47dc-ba30-24b9923b233a\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") "
Jul 10 02:46:44.731354 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.731296    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-tmp-dir\") pod \"29910a03-656c-47dc-ba30-24b9923b233a\" (UID: \"29910a03-656c-47dc-ba30-24b9923b233a\") "
Jul 10 02:46:44.731600 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.731435    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-home" (OuterVolumeSpecName: "home") pod "29910a03-656c-47dc-ba30-24b9923b233a" (UID: "29910a03-656c-47dc-ba30-24b9923b233a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:46:44.731600 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.731451    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-model-cache" (OuterVolumeSpecName: "model-cache") pod "29910a03-656c-47dc-ba30-24b9923b233a" (UID: "29910a03-656c-47dc-ba30-24b9923b233a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:46:44.731600 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.731586    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "29910a03-656c-47dc-ba30-24b9923b233a" (UID: "29910a03-656c-47dc-ba30-24b9923b233a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:46:44.731733 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.731662    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-home\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:46:44.731733 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.731682    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-model-cache\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:46:44.733587 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.733566    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29910a03-656c-47dc-ba30-24b9923b233a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "29910a03-656c-47dc-ba30-24b9923b233a" (UID: "29910a03-656c-47dc-ba30-24b9923b233a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:46:44.733662 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.733568    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-dshm" (OuterVolumeSpecName: "dshm") pod "29910a03-656c-47dc-ba30-24b9923b233a" (UID: "29910a03-656c-47dc-ba30-24b9923b233a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:46:44.733701 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.733655    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29910a03-656c-47dc-ba30-24b9923b233a-kube-api-access-xqkzj" (OuterVolumeSpecName: "kube-api-access-xqkzj") pod "29910a03-656c-47dc-ba30-24b9923b233a" (UID: "29910a03-656c-47dc-ba30-24b9923b233a"). InnerVolumeSpecName "kube-api-access-xqkzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:46:44.832647 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.832565    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-tmp-dir\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:46:44.832647 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.832594    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29910a03-656c-47dc-ba30-24b9923b233a-dshm\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:46:44.832647 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.832606    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqkzj\" (UniqueName: \"kubernetes.io/projected/29910a03-656c-47dc-ba30-24b9923b233a-kube-api-access-xqkzj\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:46:44.832647 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:44.832618    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29910a03-656c-47dc-ba30-24b9923b233a-tls-certs\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:46:45.234215 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:45.234180    2579 generic.go:356] "Generic (PLEG): container finished" podID="29910a03-656c-47dc-ba30-24b9923b233a" containerID="f50d706a3c78b6b1e59d00451d52e3df3edd079322822f2f28e14bc790e41a7d" exitCode=0
Jul 10 02:46:45.234417 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:45.234242    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"
Jul 10 02:46:45.234417 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:45.234281    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7" event={"ID":"29910a03-656c-47dc-ba30-24b9923b233a","Type":"ContainerDied","Data":"f50d706a3c78b6b1e59d00451d52e3df3edd079322822f2f28e14bc790e41a7d"}
Jul 10 02:46:45.234417 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:45.234323    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7" event={"ID":"29910a03-656c-47dc-ba30-24b9923b233a","Type":"ContainerDied","Data":"cb61032bf8213c70fe04359a0409631e2bb66d9e5588fba068e0b6c6cb70d1bc"}
Jul 10 02:46:45.234417 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:45.234339    2579 scope.go:117] "RemoveContainer" containerID="f50d706a3c78b6b1e59d00451d52e3df3edd079322822f2f28e14bc790e41a7d"
Jul 10 02:46:45.243491 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:45.243472    2579 scope.go:117] "RemoveContainer" containerID="f50d706a3c78b6b1e59d00451d52e3df3edd079322822f2f28e14bc790e41a7d"
Jul 10 02:46:45.243817 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:46:45.243799    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f50d706a3c78b6b1e59d00451d52e3df3edd079322822f2f28e14bc790e41a7d\": container with ID starting with f50d706a3c78b6b1e59d00451d52e3df3edd079322822f2f28e14bc790e41a7d not found: ID does not exist" containerID="f50d706a3c78b6b1e59d00451d52e3df3edd079322822f2f28e14bc790e41a7d"
Jul 10 02:46:45.243881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:45.243825    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f50d706a3c78b6b1e59d00451d52e3df3edd079322822f2f28e14bc790e41a7d"} err="failed to get container status \"f50d706a3c78b6b1e59d00451d52e3df3edd079322822f2f28e14bc790e41a7d\": rpc error: code = NotFound desc = could not find container \"f50d706a3c78b6b1e59d00451d52e3df3edd079322822f2f28e14bc790e41a7d\": container with ID starting with f50d706a3c78b6b1e59d00451d52e3df3edd079322822f2f28e14bc790e41a7d not found: ID does not exist"
Jul 10 02:46:45.254673 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:45.254651    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"]
Jul 10 02:46:45.258082 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:45.258062    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-1238eb/test-model-1238eb-kserve-79f88955f-mrqt7"]
Jul 10 02:46:45.566211 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:45.566136    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29910a03-656c-47dc-ba30-24b9923b233a" path="/var/lib/kubelet/pods/29910a03-656c-47dc-ba30-24b9923b233a/volumes"
Jul 10 02:46:45.566529 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:46:45.566499    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dca0d150-61c7-48cd-8186-928c6e52b437" path="/var/lib/kubelet/pods/dca0d150-61c7-48cd-8186-928c6e52b437/volumes"
Jul 10 02:47:13.724198 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:13.724124    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"]
Jul 10 02:47:13.724710 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:13.724340    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms" podUID="57e22f70-58f7-4b66-abd6-fb946a667995" containerName="maas-api" containerID="cri-o://04bfdd9e9aafd2709d657b59c915273e75f6e76af631f23c55e40fdd0dd60901" gracePeriod=30
Jul 10 02:47:13.970233 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:13.970209    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"
Jul 10 02:47:14.045588 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.045496    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57e22f70-58f7-4b66-abd6-fb946a667995-maas-api-tls\") pod \"57e22f70-58f7-4b66-abd6-fb946a667995\" (UID: \"57e22f70-58f7-4b66-abd6-fb946a667995\") "
Jul 10 02:47:14.045588 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.045571    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjph7\" (UniqueName: \"kubernetes.io/projected/57e22f70-58f7-4b66-abd6-fb946a667995-kube-api-access-vjph7\") pod \"57e22f70-58f7-4b66-abd6-fb946a667995\" (UID: \"57e22f70-58f7-4b66-abd6-fb946a667995\") "
Jul 10 02:47:14.045786 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.045622    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57e22f70-58f7-4b66-abd6-fb946a667995-openshift-service-ca\") pod \"57e22f70-58f7-4b66-abd6-fb946a667995\" (UID: \"57e22f70-58f7-4b66-abd6-fb946a667995\") "
Jul 10 02:47:14.045966 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.045941    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57e22f70-58f7-4b66-abd6-fb946a667995-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "57e22f70-58f7-4b66-abd6-fb946a667995" (UID: "57e22f70-58f7-4b66-abd6-fb946a667995"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:47:14.047789 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.047763    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57e22f70-58f7-4b66-abd6-fb946a667995-kube-api-access-vjph7" (OuterVolumeSpecName: "kube-api-access-vjph7") pod "57e22f70-58f7-4b66-abd6-fb946a667995" (UID: "57e22f70-58f7-4b66-abd6-fb946a667995"). InnerVolumeSpecName "kube-api-access-vjph7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:47:14.047883 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.047787    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57e22f70-58f7-4b66-abd6-fb946a667995-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "57e22f70-58f7-4b66-abd6-fb946a667995" (UID: "57e22f70-58f7-4b66-abd6-fb946a667995"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:47:14.146802 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.146771    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57e22f70-58f7-4b66-abd6-fb946a667995-maas-api-tls\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:47:14.146802 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.146797    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjph7\" (UniqueName: \"kubernetes.io/projected/57e22f70-58f7-4b66-abd6-fb946a667995-kube-api-access-vjph7\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:47:14.146802 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.146807    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57e22f70-58f7-4b66-abd6-fb946a667995-openshift-service-ca\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:47:14.351566 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.351535    2579 generic.go:356] "Generic (PLEG): container finished" podID="57e22f70-58f7-4b66-abd6-fb946a667995" containerID="04bfdd9e9aafd2709d657b59c915273e75f6e76af631f23c55e40fdd0dd60901" exitCode=0
Jul 10 02:47:14.351718 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.351572    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms" event={"ID":"57e22f70-58f7-4b66-abd6-fb946a667995","Type":"ContainerDied","Data":"04bfdd9e9aafd2709d657b59c915273e75f6e76af631f23c55e40fdd0dd60901"}
Jul 10 02:47:14.351718 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.351624    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms" event={"ID":"57e22f70-58f7-4b66-abd6-fb946a667995","Type":"ContainerDied","Data":"209933d79d7d192c936ca7b794f279691ccf09941d41a41fd63035383ce97902"}
Jul 10 02:47:14.351718 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.351639    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"
Jul 10 02:47:14.351718 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.351653    2579 scope.go:117] "RemoveContainer" containerID="04bfdd9e9aafd2709d657b59c915273e75f6e76af631f23c55e40fdd0dd60901"
Jul 10 02:47:14.359965 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.359944    2579 scope.go:117] "RemoveContainer" containerID="04bfdd9e9aafd2709d657b59c915273e75f6e76af631f23c55e40fdd0dd60901"
Jul 10 02:47:14.360236 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:47:14.360214    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04bfdd9e9aafd2709d657b59c915273e75f6e76af631f23c55e40fdd0dd60901\": container with ID starting with 04bfdd9e9aafd2709d657b59c915273e75f6e76af631f23c55e40fdd0dd60901 not found: ID does not exist" containerID="04bfdd9e9aafd2709d657b59c915273e75f6e76af631f23c55e40fdd0dd60901"
Jul 10 02:47:14.360297 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.360243    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04bfdd9e9aafd2709d657b59c915273e75f6e76af631f23c55e40fdd0dd60901"} err="failed to get container status \"04bfdd9e9aafd2709d657b59c915273e75f6e76af631f23c55e40fdd0dd60901\": rpc error: code = NotFound desc = could not find container \"04bfdd9e9aafd2709d657b59c915273e75f6e76af631f23c55e40fdd0dd60901\": container with ID starting with 04bfdd9e9aafd2709d657b59c915273e75f6e76af631f23c55e40fdd0dd60901 not found: ID does not exist"
Jul 10 02:47:14.380460 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.380437    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"]
Jul 10 02:47:14.391447 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:14.391426    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1238eb-cdff86f45-qqbms"]
Jul 10 02:47:15.565803 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:15.565770    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57e22f70-58f7-4b66-abd6-fb946a667995" path="/var/lib/kubelet/pods/57e22f70-58f7-4b66-abd6-fb946a667995/volumes"
Jul 10 02:47:16.355159 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:16.355129    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:47:23.356852 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.356783    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"]
Jul 10 02:47:23.357344 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.357206    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57e22f70-58f7-4b66-abd6-fb946a667995" containerName="maas-api"
Jul 10 02:47:23.357344 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.357224    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="57e22f70-58f7-4b66-abd6-fb946a667995" containerName="maas-api"
Jul 10 02:47:23.357344 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.357241    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29910a03-656c-47dc-ba30-24b9923b233a" containerName="main"
Jul 10 02:47:23.357344 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.357248    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="29910a03-656c-47dc-ba30-24b9923b233a" containerName="main"
Jul 10 02:47:23.357344 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.357264    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dca0d150-61c7-48cd-8186-928c6e52b437" containerName="main"
Jul 10 02:47:23.357344 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.357273    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dca0d150-61c7-48cd-8186-928c6e52b437" containerName="main"
Jul 10 02:47:23.357344 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.357285    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="594104f2-dc53-4d7d-afde-cf888545bd4b" containerName="maas-api"
Jul 10 02:47:23.357344 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.357293    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="594104f2-dc53-4d7d-afde-cf888545bd4b" containerName="maas-api"
Jul 10 02:47:23.357738 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.357364    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="57e22f70-58f7-4b66-abd6-fb946a667995" containerName="maas-api"
Jul 10 02:47:23.357738 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.357376    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="594104f2-dc53-4d7d-afde-cf888545bd4b" containerName="maas-api"
Jul 10 02:47:23.357738 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.357385    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="29910a03-656c-47dc-ba30-24b9923b233a" containerName="main"
Jul 10 02:47:23.357738 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.357392    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dca0d150-61c7-48cd-8186-928c6e52b437" containerName="main"
Jul 10 02:47:23.360430 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.360411    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"
Jul 10 02:47:23.363157 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.363137    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 02:47:23.363283 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.363173    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-c90820-serving-cert\""
Jul 10 02:47:23.363283 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.363181    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 02:47:23.364233 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.364194    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-scvcw\""
Jul 10 02:47:23.370477 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.370452    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"]
Jul 10 02:47:23.418739 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.418710    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf7c534d-6572-4869-8b07-6cc3711b726c-openshift-service-ca\") pod \"maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g\" (UID: \"bf7c534d-6572-4869-8b07-6cc3711b726c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"
Jul 10 02:47:23.418916 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.418753    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf7c534d-6572-4869-8b07-6cc3711b726c-maas-api-tls\") pod \"maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g\" (UID: \"bf7c534d-6572-4869-8b07-6cc3711b726c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"
Jul 10 02:47:23.418916 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.418779    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xsjh\" (UniqueName: \"kubernetes.io/projected/bf7c534d-6572-4869-8b07-6cc3711b726c-kube-api-access-6xsjh\") pod \"maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g\" (UID: \"bf7c534d-6572-4869-8b07-6cc3711b726c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"
Jul 10 02:47:23.519288 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.519256    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf7c534d-6572-4869-8b07-6cc3711b726c-openshift-service-ca\") pod \"maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g\" (UID: \"bf7c534d-6572-4869-8b07-6cc3711b726c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"
Jul 10 02:47:23.519447 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.519297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf7c534d-6572-4869-8b07-6cc3711b726c-maas-api-tls\") pod \"maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g\" (UID: \"bf7c534d-6572-4869-8b07-6cc3711b726c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"
Jul 10 02:47:23.519447 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.519324    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xsjh\" (UniqueName: \"kubernetes.io/projected/bf7c534d-6572-4869-8b07-6cc3711b726c-kube-api-access-6xsjh\") pod \"maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g\" (UID: \"bf7c534d-6572-4869-8b07-6cc3711b726c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"
Jul 10 02:47:23.519992 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.519974    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf7c534d-6572-4869-8b07-6cc3711b726c-openshift-service-ca\") pod \"maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g\" (UID: \"bf7c534d-6572-4869-8b07-6cc3711b726c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"
Jul 10 02:47:23.521865 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.521845    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf7c534d-6572-4869-8b07-6cc3711b726c-maas-api-tls\") pod \"maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g\" (UID: \"bf7c534d-6572-4869-8b07-6cc3711b726c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"
Jul 10 02:47:23.533915 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.533885    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xsjh\" (UniqueName: \"kubernetes.io/projected/bf7c534d-6572-4869-8b07-6cc3711b726c-kube-api-access-6xsjh\") pod \"maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g\" (UID: \"bf7c534d-6572-4869-8b07-6cc3711b726c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"
Jul 10 02:47:23.672959 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.672856    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"
Jul 10 02:47:23.816248 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:23.816225    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"]
Jul 10 02:47:23.818773 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:47:23.818742    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf7c534d_6572_4869_8b07_6cc3711b726c.slice/crio-840e0f28c9ecda7cd67201aa3ac2e52922b673aa240a58e68cba0729897b4f89 WatchSource:0}: Error finding container 840e0f28c9ecda7cd67201aa3ac2e52922b673aa240a58e68cba0729897b4f89: Status 404 returned error can't find the container with id 840e0f28c9ecda7cd67201aa3ac2e52922b673aa240a58e68cba0729897b4f89
Jul 10 02:47:24.392084 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:24.392050    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g" event={"ID":"bf7c534d-6572-4869-8b07-6cc3711b726c","Type":"ContainerStarted","Data":"f8522623fcfeb336a8b9b3cb92e1c4ce283b2faee3274eb009361bd3773aa655"}
Jul 10 02:47:24.392536 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:24.392102    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g" event={"ID":"bf7c534d-6572-4869-8b07-6cc3711b726c","Type":"ContainerStarted","Data":"840e0f28c9ecda7cd67201aa3ac2e52922b673aa240a58e68cba0729897b4f89"}
Jul 10 02:47:24.392536 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:24.392137    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"
Jul 10 02:47:24.408989 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:24.408942    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g" podStartSLOduration=0.963406292 podStartE2EDuration="1.408926082s" totalImagesPullingTime="445.51979ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:47:24.408560442 +0000 UTC m=+1991.391318129" watchObservedRunningTime="2026-07-10 02:47:24.408926082 +0000 UTC m=+1991.391683767"
Jul 10 02:47:30.401765 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:30.401736    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"
Jul 10 02:47:31.364915 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:31.364881    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:47:31.386326 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:31.386297    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"]
Jul 10 02:47:31.389555 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:31.389536    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:31.392475 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:31.392454    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f312f7-serving-cert\""
Jul 10 02:47:31.401633 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:31.401612    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"]
Jul 10 02:47:31.483687 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:31.483636    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67fa7822-2e31-4063-9154-954abec26e28-openshift-service-ca\") pod \"maas-api-e2e-api-b-f312f7-6555bb75d-swj6h\" (UID: \"67fa7822-2e31-4063-9154-954abec26e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:31.484115 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:31.483701    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2bt2\" (UniqueName: \"kubernetes.io/projected/67fa7822-2e31-4063-9154-954abec26e28-kube-api-access-r2bt2\") pod \"maas-api-e2e-api-b-f312f7-6555bb75d-swj6h\" (UID: \"67fa7822-2e31-4063-9154-954abec26e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:31.484115 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:31.483750    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67fa7822-2e31-4063-9154-954abec26e28-maas-api-tls\") pod \"maas-api-e2e-api-b-f312f7-6555bb75d-swj6h\" (UID: \"67fa7822-2e31-4063-9154-954abec26e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:31.584711 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:31.584668    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67fa7822-2e31-4063-9154-954abec26e28-maas-api-tls\") pod \"maas-api-e2e-api-b-f312f7-6555bb75d-swj6h\" (UID: \"67fa7822-2e31-4063-9154-954abec26e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:31.584877 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:31.584752    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67fa7822-2e31-4063-9154-954abec26e28-openshift-service-ca\") pod \"maas-api-e2e-api-b-f312f7-6555bb75d-swj6h\" (UID: \"67fa7822-2e31-4063-9154-954abec26e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:31.584877 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:31.584795    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2bt2\" (UniqueName: \"kubernetes.io/projected/67fa7822-2e31-4063-9154-954abec26e28-kube-api-access-r2bt2\") pod \"maas-api-e2e-api-b-f312f7-6555bb75d-swj6h\" (UID: \"67fa7822-2e31-4063-9154-954abec26e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:31.584877 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:47:31.584831    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-serving-cert: secret "maas-api-e2e-api-b-f312f7-serving-cert" not found
Jul 10 02:47:31.585063 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:47:31.584895    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67fa7822-2e31-4063-9154-954abec26e28-maas-api-tls podName:67fa7822-2e31-4063-9154-954abec26e28 nodeName:}" failed. No retries permitted until 2026-07-10 02:47:32.084875118 +0000 UTC m=+1999.067632800 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/67fa7822-2e31-4063-9154-954abec26e28-maas-api-tls") pod "maas-api-e2e-api-b-f312f7-6555bb75d-swj6h" (UID: "67fa7822-2e31-4063-9154-954abec26e28") : secret "maas-api-e2e-api-b-f312f7-serving-cert" not found
Jul 10 02:47:31.585440 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:31.585417    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67fa7822-2e31-4063-9154-954abec26e28-openshift-service-ca\") pod \"maas-api-e2e-api-b-f312f7-6555bb75d-swj6h\" (UID: \"67fa7822-2e31-4063-9154-954abec26e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:31.598912 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:31.598877    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2bt2\" (UniqueName: \"kubernetes.io/projected/67fa7822-2e31-4063-9154-954abec26e28-kube-api-access-r2bt2\") pod \"maas-api-e2e-api-b-f312f7-6555bb75d-swj6h\" (UID: \"67fa7822-2e31-4063-9154-954abec26e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:32.090590 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:32.090557    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67fa7822-2e31-4063-9154-954abec26e28-maas-api-tls\") pod \"maas-api-e2e-api-b-f312f7-6555bb75d-swj6h\" (UID: \"67fa7822-2e31-4063-9154-954abec26e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:32.093364 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:32.093340    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67fa7822-2e31-4063-9154-954abec26e28-maas-api-tls\") pod \"maas-api-e2e-api-b-f312f7-6555bb75d-swj6h\" (UID: \"67fa7822-2e31-4063-9154-954abec26e28\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:32.300399 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:32.300363    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:32.434033 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:32.433999    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"]
Jul 10 02:47:32.438142 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:47:32.438115    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67fa7822_2e31_4063_9154_954abec26e28.slice/crio-83eb3e57677ce224a367edd2ddaf170f41d5ebcb1acd3fefafdb01c703de57a8 WatchSource:0}: Error finding container 83eb3e57677ce224a367edd2ddaf170f41d5ebcb1acd3fefafdb01c703de57a8: Status 404 returned error can't find the container with id 83eb3e57677ce224a367edd2ddaf170f41d5ebcb1acd3fefafdb01c703de57a8
Jul 10 02:47:33.431196 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:33.431162    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h" event={"ID":"67fa7822-2e31-4063-9154-954abec26e28","Type":"ContainerStarted","Data":"2021cd6d94515c22d0e6594ba57cd35faa3c1969442ef8bf812bbca49e8468cc"}
Jul 10 02:47:33.431598 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:33.431221    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h" event={"ID":"67fa7822-2e31-4063-9154-954abec26e28","Type":"ContainerStarted","Data":"83eb3e57677ce224a367edd2ddaf170f41d5ebcb1acd3fefafdb01c703de57a8"}
Jul 10 02:47:33.431598 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:33.431273    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:33.448931 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:33.448887    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h" podStartSLOduration=2.058560207 podStartE2EDuration="2.448872765s" totalImagesPullingTime="390.312558ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:47:33.447388539 +0000 UTC m=+2000.430146226" watchObservedRunningTime="2026-07-10 02:47:33.448872765 +0000 UTC m=+2000.431630478"
Jul 10 02:47:39.440893 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:39.440861    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:44.354602 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:44.354569    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:47:44.770517 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:44.770423    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"]
Jul 10 02:47:44.770747 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:44.770679    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g" podUID="bf7c534d-6572-4869-8b07-6cc3711b726c" containerName="maas-api" containerID="cri-o://f8522623fcfeb336a8b9b3cb92e1c4ce283b2faee3274eb009361bd3773aa655" gracePeriod=30
Jul 10 02:47:45.013338 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.013314    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"
Jul 10 02:47:45.096907 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.096831    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xsjh\" (UniqueName: \"kubernetes.io/projected/bf7c534d-6572-4869-8b07-6cc3711b726c-kube-api-access-6xsjh\") pod \"bf7c534d-6572-4869-8b07-6cc3711b726c\" (UID: \"bf7c534d-6572-4869-8b07-6cc3711b726c\") "
Jul 10 02:47:45.097040 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.096922    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf7c534d-6572-4869-8b07-6cc3711b726c-maas-api-tls\") pod \"bf7c534d-6572-4869-8b07-6cc3711b726c\" (UID: \"bf7c534d-6572-4869-8b07-6cc3711b726c\") "
Jul 10 02:47:45.097040 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.096980    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf7c534d-6572-4869-8b07-6cc3711b726c-openshift-service-ca\") pod \"bf7c534d-6572-4869-8b07-6cc3711b726c\" (UID: \"bf7c534d-6572-4869-8b07-6cc3711b726c\") "
Jul 10 02:47:45.097340 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.097318    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf7c534d-6572-4869-8b07-6cc3711b726c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bf7c534d-6572-4869-8b07-6cc3711b726c" (UID: "bf7c534d-6572-4869-8b07-6cc3711b726c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:47:45.099057 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.099035    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf7c534d-6572-4869-8b07-6cc3711b726c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bf7c534d-6572-4869-8b07-6cc3711b726c" (UID: "bf7c534d-6572-4869-8b07-6cc3711b726c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:47:45.099139 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.099120    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf7c534d-6572-4869-8b07-6cc3711b726c-kube-api-access-6xsjh" (OuterVolumeSpecName: "kube-api-access-6xsjh") pod "bf7c534d-6572-4869-8b07-6cc3711b726c" (UID: "bf7c534d-6572-4869-8b07-6cc3711b726c"). InnerVolumeSpecName "kube-api-access-6xsjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:47:45.198008 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.197973    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf7c534d-6572-4869-8b07-6cc3711b726c-openshift-service-ca\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:47:45.198008 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.198002    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xsjh\" (UniqueName: \"kubernetes.io/projected/bf7c534d-6572-4869-8b07-6cc3711b726c-kube-api-access-6xsjh\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:47:45.198008 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.198012    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf7c534d-6572-4869-8b07-6cc3711b726c-maas-api-tls\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:47:45.479544 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.479495    2579 generic.go:356] "Generic (PLEG): container finished" podID="bf7c534d-6572-4869-8b07-6cc3711b726c" containerID="f8522623fcfeb336a8b9b3cb92e1c4ce283b2faee3274eb009361bd3773aa655" exitCode=0
Jul 10 02:47:45.480013 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.479539    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g" event={"ID":"bf7c534d-6572-4869-8b07-6cc3711b726c","Type":"ContainerDied","Data":"f8522623fcfeb336a8b9b3cb92e1c4ce283b2faee3274eb009361bd3773aa655"}
Jul 10 02:47:45.480013 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.479608    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g" event={"ID":"bf7c534d-6572-4869-8b07-6cc3711b726c","Type":"ContainerDied","Data":"840e0f28c9ecda7cd67201aa3ac2e52922b673aa240a58e68cba0729897b4f89"}
Jul 10 02:47:45.480013 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.479556    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"
Jul 10 02:47:45.480013 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.479638    2579 scope.go:117] "RemoveContainer" containerID="f8522623fcfeb336a8b9b3cb92e1c4ce283b2faee3274eb009361bd3773aa655"
Jul 10 02:47:45.488539 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.488499    2579 scope.go:117] "RemoveContainer" containerID="f8522623fcfeb336a8b9b3cb92e1c4ce283b2faee3274eb009361bd3773aa655"
Jul 10 02:47:45.488823 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:47:45.488805    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8522623fcfeb336a8b9b3cb92e1c4ce283b2faee3274eb009361bd3773aa655\": container with ID starting with f8522623fcfeb336a8b9b3cb92e1c4ce283b2faee3274eb009361bd3773aa655 not found: ID does not exist" containerID="f8522623fcfeb336a8b9b3cb92e1c4ce283b2faee3274eb009361bd3773aa655"
Jul 10 02:47:45.488894 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.488831    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8522623fcfeb336a8b9b3cb92e1c4ce283b2faee3274eb009361bd3773aa655"} err="failed to get container status \"f8522623fcfeb336a8b9b3cb92e1c4ce283b2faee3274eb009361bd3773aa655\": rpc error: code = NotFound desc = could not find container \"f8522623fcfeb336a8b9b3cb92e1c4ce283b2faee3274eb009361bd3773aa655\": container with ID starting with f8522623fcfeb336a8b9b3cb92e1c4ce283b2faee3274eb009361bd3773aa655 not found: ID does not exist"
Jul 10 02:47:45.500274 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.500249    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"]
Jul 10 02:47:45.504554 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.504534    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c90820-7f49fc78f6-26q4g"]
Jul 10 02:47:45.566181 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:45.566154    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf7c534d-6572-4869-8b07-6cc3711b726c" path="/var/lib/kubelet/pods/bf7c534d-6572-4869-8b07-6cc3711b726c/volumes"
Jul 10 02:47:52.125342 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.125310    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"]
Jul 10 02:47:52.125759 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.125532    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h" podUID="67fa7822-2e31-4063-9154-954abec26e28" containerName="maas-api" containerID="cri-o://2021cd6d94515c22d0e6594ba57cd35faa3c1969442ef8bf812bbca49e8468cc" gracePeriod=30
Jul 10 02:47:52.367474 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.367449    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:52.455684 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.455600    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67fa7822-2e31-4063-9154-954abec26e28-maas-api-tls\") pod \"67fa7822-2e31-4063-9154-954abec26e28\" (UID: \"67fa7822-2e31-4063-9154-954abec26e28\") "
Jul 10 02:47:52.455684 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.455658    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2bt2\" (UniqueName: \"kubernetes.io/projected/67fa7822-2e31-4063-9154-954abec26e28-kube-api-access-r2bt2\") pod \"67fa7822-2e31-4063-9154-954abec26e28\" (UID: \"67fa7822-2e31-4063-9154-954abec26e28\") "
Jul 10 02:47:52.455684 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.455687    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67fa7822-2e31-4063-9154-954abec26e28-openshift-service-ca\") pod \"67fa7822-2e31-4063-9154-954abec26e28\" (UID: \"67fa7822-2e31-4063-9154-954abec26e28\") "
Jul 10 02:47:52.456124 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.456093    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67fa7822-2e31-4063-9154-954abec26e28-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "67fa7822-2e31-4063-9154-954abec26e28" (UID: "67fa7822-2e31-4063-9154-954abec26e28"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:47:52.457867 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.457843    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67fa7822-2e31-4063-9154-954abec26e28-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "67fa7822-2e31-4063-9154-954abec26e28" (UID: "67fa7822-2e31-4063-9154-954abec26e28"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:47:52.457987 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.457968    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67fa7822-2e31-4063-9154-954abec26e28-kube-api-access-r2bt2" (OuterVolumeSpecName: "kube-api-access-r2bt2") pod "67fa7822-2e31-4063-9154-954abec26e28" (UID: "67fa7822-2e31-4063-9154-954abec26e28"). InnerVolumeSpecName "kube-api-access-r2bt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:47:52.510212 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.510183    2579 generic.go:356] "Generic (PLEG): container finished" podID="67fa7822-2e31-4063-9154-954abec26e28" containerID="2021cd6d94515c22d0e6594ba57cd35faa3c1969442ef8bf812bbca49e8468cc" exitCode=0
Jul 10 02:47:52.510366 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.510224    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h" event={"ID":"67fa7822-2e31-4063-9154-954abec26e28","Type":"ContainerDied","Data":"2021cd6d94515c22d0e6594ba57cd35faa3c1969442ef8bf812bbca49e8468cc"}
Jul 10 02:47:52.510366 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.510229    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"
Jul 10 02:47:52.510366 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.510259    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h" event={"ID":"67fa7822-2e31-4063-9154-954abec26e28","Type":"ContainerDied","Data":"83eb3e57677ce224a367edd2ddaf170f41d5ebcb1acd3fefafdb01c703de57a8"}
Jul 10 02:47:52.510366 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.510279    2579 scope.go:117] "RemoveContainer" containerID="2021cd6d94515c22d0e6594ba57cd35faa3c1969442ef8bf812bbca49e8468cc"
Jul 10 02:47:52.518614 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.518598    2579 scope.go:117] "RemoveContainer" containerID="2021cd6d94515c22d0e6594ba57cd35faa3c1969442ef8bf812bbca49e8468cc"
Jul 10 02:47:52.518848 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:47:52.518831    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2021cd6d94515c22d0e6594ba57cd35faa3c1969442ef8bf812bbca49e8468cc\": container with ID starting with 2021cd6d94515c22d0e6594ba57cd35faa3c1969442ef8bf812bbca49e8468cc not found: ID does not exist" containerID="2021cd6d94515c22d0e6594ba57cd35faa3c1969442ef8bf812bbca49e8468cc"
Jul 10 02:47:52.518891 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.518856    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2021cd6d94515c22d0e6594ba57cd35faa3c1969442ef8bf812bbca49e8468cc"} err="failed to get container status \"2021cd6d94515c22d0e6594ba57cd35faa3c1969442ef8bf812bbca49e8468cc\": rpc error: code = NotFound desc = could not find container \"2021cd6d94515c22d0e6594ba57cd35faa3c1969442ef8bf812bbca49e8468cc\": container with ID starting with 2021cd6d94515c22d0e6594ba57cd35faa3c1969442ef8bf812bbca49e8468cc not found: ID does not exist"
Jul 10 02:47:52.530590 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.530565    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"]
Jul 10 02:47:52.534450 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.534429    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f312f7-6555bb75d-swj6h"]
Jul 10 02:47:52.556499 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.556477    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2bt2\" (UniqueName: \"kubernetes.io/projected/67fa7822-2e31-4063-9154-954abec26e28-kube-api-access-r2bt2\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:47:52.556583 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.556497    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67fa7822-2e31-4063-9154-954abec26e28-openshift-service-ca\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:47:52.556583 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:52.556526    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67fa7822-2e31-4063-9154-954abec26e28-maas-api-tls\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:47:53.565370 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:53.565333    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67fa7822-2e31-4063-9154-954abec26e28" path="/var/lib/kubelet/pods/67fa7822-2e31-4063-9154-954abec26e28/volumes"
Jul 10 02:47:54.458344 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:47:54.458307    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:48:02.566486 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.566453    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"]
Jul 10 02:48:02.566878 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.566862    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67fa7822-2e31-4063-9154-954abec26e28" containerName="maas-api"
Jul 10 02:48:02.566924 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.566881    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="67fa7822-2e31-4063-9154-954abec26e28" containerName="maas-api"
Jul 10 02:48:02.566924 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.566913    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf7c534d-6572-4869-8b07-6cc3711b726c" containerName="maas-api"
Jul 10 02:48:02.566924 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.566922    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf7c534d-6572-4869-8b07-6cc3711b726c" containerName="maas-api"
Jul 10 02:48:02.567010 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.566993    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="67fa7822-2e31-4063-9154-954abec26e28" containerName="maas-api"
Jul 10 02:48:02.567010 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.567006    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf7c534d-6572-4869-8b07-6cc3711b726c" containerName="maas-api"
Jul 10 02:48:02.571384 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.571367    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"
Jul 10 02:48:02.574053 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.574028    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 02:48:02.575200 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.575179    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-e239ac-serving-cert\""
Jul 10 02:48:02.575200 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.575193    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-scvcw\""
Jul 10 02:48:02.575327 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.575187    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 02:48:02.580538 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.580495    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"]
Jul 10 02:48:02.736271 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.736236    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-maas-api-tls\") pod \"maas-api-e2e-shared-a-e239ac-677458755b-wdbpx\" (UID: \"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"
Jul 10 02:48:02.736446 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.736289    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e239ac-677458755b-wdbpx\" (UID: \"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"
Jul 10 02:48:02.736446 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.736371    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tb25\" (UniqueName: \"kubernetes.io/projected/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-kube-api-access-4tb25\") pod \"maas-api-e2e-shared-a-e239ac-677458755b-wdbpx\" (UID: \"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"
Jul 10 02:48:02.837398 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.837308    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-maas-api-tls\") pod \"maas-api-e2e-shared-a-e239ac-677458755b-wdbpx\" (UID: \"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"
Jul 10 02:48:02.837398 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.837357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e239ac-677458755b-wdbpx\" (UID: \"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"
Jul 10 02:48:02.837398 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.837391    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tb25\" (UniqueName: \"kubernetes.io/projected/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-kube-api-access-4tb25\") pod \"maas-api-e2e-shared-a-e239ac-677458755b-wdbpx\" (UID: \"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"
Jul 10 02:48:02.838046 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.838026    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e239ac-677458755b-wdbpx\" (UID: \"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"
Jul 10 02:48:02.839805 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.839788    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-maas-api-tls\") pod \"maas-api-e2e-shared-a-e239ac-677458755b-wdbpx\" (UID: \"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"
Jul 10 02:48:02.845062 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.845038    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tb25\" (UniqueName: \"kubernetes.io/projected/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-kube-api-access-4tb25\") pod \"maas-api-e2e-shared-a-e239ac-677458755b-wdbpx\" (UID: \"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"
Jul 10 02:48:02.883338 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:02.883307    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"
Jul 10 02:48:03.012360 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:03.012333    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"]
Jul 10 02:48:03.014881 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:48:03.014856    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ea8e0a5_dd16_4ba1_82e7_9cb41a53b1d0.slice/crio-583ea3c25e0dbdeacb0e75b198158f2446183f019a75a94b7c8b7d0b672ca80a WatchSource:0}: Error finding container 583ea3c25e0dbdeacb0e75b198158f2446183f019a75a94b7c8b7d0b672ca80a: Status 404 returned error can't find the container with id 583ea3c25e0dbdeacb0e75b198158f2446183f019a75a94b7c8b7d0b672ca80a
Jul 10 02:48:03.554656 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:03.554620    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx" event={"ID":"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0","Type":"ContainerStarted","Data":"ea69ad4694f2bc786e01848e2a3de5dff7272ec91cb457caae842e1de09e92d9"}
Jul 10 02:48:03.554810 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:03.554676    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx" event={"ID":"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0","Type":"ContainerStarted","Data":"583ea3c25e0dbdeacb0e75b198158f2446183f019a75a94b7c8b7d0b672ca80a"}
Jul 10 02:48:03.554810 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:03.554747    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"
Jul 10 02:48:03.572158 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:03.572087    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx" podStartSLOduration=1.223158136 podStartE2EDuration="1.572060961s" totalImagesPullingTime="348.902825ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:48:03.571641702 +0000 UTC m=+2030.554399390" watchObservedRunningTime="2026-07-10 02:48:03.572060961 +0000 UTC m=+2030.554818638"
Jul 10 02:48:08.552453 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:08.552411    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:48:09.570742 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:09.570716    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"
Jul 10 02:48:10.743067 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:10.743034    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"]
Jul 10 02:48:10.746882 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:10.746860    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:48:10.749746 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:10.749726    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-e812a4-serving-cert\""
Jul 10 02:48:10.756163 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:10.756138    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"]
Jul 10 02:48:10.806307 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:10.806275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0601606-d77f-4c2f-9f90-b99fe089a08f-maas-api-tls\") pod \"maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz\" (UID: \"d0601606-d77f-4c2f-9f90-b99fe089a08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:48:10.806440 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:10.806344    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0601606-d77f-4c2f-9f90-b99fe089a08f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz\" (UID: \"d0601606-d77f-4c2f-9f90-b99fe089a08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:48:10.806440 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:10.806373    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sqlw\" (UniqueName: \"kubernetes.io/projected/d0601606-d77f-4c2f-9f90-b99fe089a08f-kube-api-access-8sqlw\") pod \"maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz\" (UID: \"d0601606-d77f-4c2f-9f90-b99fe089a08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:48:10.907577 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:10.907534    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0601606-d77f-4c2f-9f90-b99fe089a08f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz\" (UID: \"d0601606-d77f-4c2f-9f90-b99fe089a08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:48:10.907749 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:10.907595    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sqlw\" (UniqueName: \"kubernetes.io/projected/d0601606-d77f-4c2f-9f90-b99fe089a08f-kube-api-access-8sqlw\") pod \"maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz\" (UID: \"d0601606-d77f-4c2f-9f90-b99fe089a08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:48:10.907749 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:10.907673    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0601606-d77f-4c2f-9f90-b99fe089a08f-maas-api-tls\") pod \"maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz\" (UID: \"d0601606-d77f-4c2f-9f90-b99fe089a08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:48:10.907829 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:48:10.907801    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-serving-cert: secret "maas-api-e2e-shared-b-e812a4-serving-cert" not found
Jul 10 02:48:10.907871 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:48:10.907866    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0601606-d77f-4c2f-9f90-b99fe089a08f-maas-api-tls podName:d0601606-d77f-4c2f-9f90-b99fe089a08f nodeName:}" failed. No retries permitted until 2026-07-10 02:48:11.407845486 +0000 UTC m=+2038.390603155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d0601606-d77f-4c2f-9f90-b99fe089a08f-maas-api-tls") pod "maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz" (UID: "d0601606-d77f-4c2f-9f90-b99fe089a08f") : secret "maas-api-e2e-shared-b-e812a4-serving-cert" not found
Jul 10 02:48:10.908188 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:10.908169    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0601606-d77f-4c2f-9f90-b99fe089a08f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz\" (UID: \"d0601606-d77f-4c2f-9f90-b99fe089a08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:48:10.916449 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:10.916428    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sqlw\" (UniqueName: \"kubernetes.io/projected/d0601606-d77f-4c2f-9f90-b99fe089a08f-kube-api-access-8sqlw\") pod \"maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz\" (UID: \"d0601606-d77f-4c2f-9f90-b99fe089a08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:48:11.411688 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:11.411642    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0601606-d77f-4c2f-9f90-b99fe089a08f-maas-api-tls\") pod \"maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz\" (UID: \"d0601606-d77f-4c2f-9f90-b99fe089a08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:48:11.414186 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:11.414158    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0601606-d77f-4c2f-9f90-b99fe089a08f-maas-api-tls\") pod \"maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz\" (UID: \"d0601606-d77f-4c2f-9f90-b99fe089a08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:48:11.661034 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:11.660991    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:48:11.792841 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:11.792702    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"]
Jul 10 02:48:11.794860 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:48:11.794834    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0601606_d77f_4c2f_9f90_b99fe089a08f.slice/crio-46f77f6c76d2cc6712d130d0001553019ae58db0ebbf1ca8d193a8d183ba35c2 WatchSource:0}: Error finding container 46f77f6c76d2cc6712d130d0001553019ae58db0ebbf1ca8d193a8d183ba35c2: Status 404 returned error can't find the container with id 46f77f6c76d2cc6712d130d0001553019ae58db0ebbf1ca8d193a8d183ba35c2
Jul 10 02:48:12.590469 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:12.590385    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz" event={"ID":"d0601606-d77f-4c2f-9f90-b99fe089a08f","Type":"ContainerStarted","Data":"c7dd4373e0ec095f63ac6f6506930274cdcd1054aafc0c5550f841146be49589"}
Jul 10 02:48:12.590469 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:12.590440    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz" event={"ID":"d0601606-d77f-4c2f-9f90-b99fe089a08f","Type":"ContainerStarted","Data":"46f77f6c76d2cc6712d130d0001553019ae58db0ebbf1ca8d193a8d183ba35c2"}
Jul 10 02:48:12.590708 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:12.590554    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:48:12.606828 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:12.606783    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz" podStartSLOduration=2.226376272 podStartE2EDuration="2.606769938s" totalImagesPullingTime="380.393666ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:48:12.606347519 +0000 UTC m=+2039.589105205" watchObservedRunningTime="2026-07-10 02:48:12.606769938 +0000 UTC m=+2039.589527623"
Jul 10 02:48:18.599841 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:18.599814    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:48:22.154287 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:22.154248    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:48:31.250708 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.250670    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"]
Jul 10 02:48:31.254245 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.254222    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.256893 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.256873    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e239ac\"/\"kube-root-ca.crt\""
Jul 10 02:48:31.258142 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.258113    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e239ac\"/\"default-dockercfg-5hv2n\""
Jul 10 02:48:31.258142 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.258123    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e239ac\"/\"openshift-service-ca.crt\""
Jul 10 02:48:31.258321 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.258216    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e239ac\"/\"auth-test-model-e239ac-kserve-self-signed-certs\""
Jul 10 02:48:31.265230 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.265210    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"]
Jul 10 02:48:31.365773 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.365732    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwxmn\" (UniqueName: \"kubernetes.io/projected/4c961630-eb30-465c-86da-41c925cafca6-kube-api-access-nwxmn\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.365962 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.365783    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-model-cache\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.365962 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.365855    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-home\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.365962 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.365877    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-dshm\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.365962 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.365906    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-tmp-dir\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.365962 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.365928    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c961630-eb30-465c-86da-41c925cafca6-tls-certs\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.467171 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.467135    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-home\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.467171 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.467170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-dshm\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.467408 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.467191    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-tmp-dir\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.467408 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.467206    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c961630-eb30-465c-86da-41c925cafca6-tls-certs\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.467408 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.467261    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwxmn\" (UniqueName: \"kubernetes.io/projected/4c961630-eb30-465c-86da-41c925cafca6-kube-api-access-nwxmn\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.467408 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.467301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-model-cache\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.467657 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.467626    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-tmp-dir\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.467722 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.467701    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-home\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.467760 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.467725    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-model-cache\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.469680 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.469657    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-dshm\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.469941 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.469920    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c961630-eb30-465c-86da-41c925cafca6-tls-certs\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.475634 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.475613    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwxmn\" (UniqueName: \"kubernetes.io/projected/4c961630-eb30-465c-86da-41c925cafca6-kube-api-access-nwxmn\") pod \"auth-test-model-e239ac-kserve-7994f94855-vwptj\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") " pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.563839 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.563765    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:31.693531 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:31.693491    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"]
Jul 10 02:48:31.695646 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:48:31.695617    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c961630_eb30_465c_86da_41c925cafca6.slice/crio-c519fa92193c2c719ffe47382c6542e4babee4a9347e309945de9c0aac0dc581 WatchSource:0}: Error finding container c519fa92193c2c719ffe47382c6542e4babee4a9347e309945de9c0aac0dc581: Status 404 returned error can't find the container with id c519fa92193c2c719ffe47382c6542e4babee4a9347e309945de9c0aac0dc581
Jul 10 02:48:32.673211 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:32.673172    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj" event={"ID":"4c961630-eb30-465c-86da-41c925cafca6","Type":"ContainerStarted","Data":"99cfb56a240e769dc7410a2faa08aef4ec6c9b9c6af6ec6b7a7901e4d694a9c9"}
Jul 10 02:48:32.673663 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:32.673217    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj" event={"ID":"4c961630-eb30-465c-86da-41c925cafca6","Type":"ContainerStarted","Data":"c519fa92193c2c719ffe47382c6542e4babee4a9347e309945de9c0aac0dc581"}
Jul 10 02:48:32.694243 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:32.694198    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj" podStartSLOduration=1.694184276 podStartE2EDuration="1.694184276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:48:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:48:32.693641227 +0000 UTC m=+2059.676398913" watchObservedRunningTime="2026-07-10 02:48:32.694184276 +0000 UTC m=+2059.676941962"
Jul 10 02:48:34.661537 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:34.661441    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:48:41.566453 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:41.566417    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:41.566943 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:41.566466    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:41.576803 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:41.576781    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:48:41.721533 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:48:41.721488    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:49:08.959422 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:08.959390    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"]
Jul 10 02:49:08.962798 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:08.962782    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:08.965586 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:08.965564    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e812a4\"/\"kube-root-ca.crt\""
Jul 10 02:49:08.966571 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:08.966549    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e812a4\"/\"default-dockercfg-j8jr4\""
Jul 10 02:49:08.966681 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:08.966553    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e812a4\"/\"openshift-service-ca.crt\""
Jul 10 02:49:08.966681 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:08.966606    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e812a4\"/\"auth-test-model-e812a4-kserve-self-signed-certs\""
Jul 10 02:49:08.971592 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:08.971571    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"]
Jul 10 02:49:09.069366 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.069324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-model-cache\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.069576 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.069403    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-dshm\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.069576 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.069472    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/633b16f1-2a54-444a-be46-a94251f0f333-tls-certs\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.069576 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.069495    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-home\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.069576 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.069539    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-644l9\" (UniqueName: \"kubernetes.io/projected/633b16f1-2a54-444a-be46-a94251f0f333-kube-api-access-644l9\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.069576 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.069561    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-tmp-dir\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.170425 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.170394    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/633b16f1-2a54-444a-be46-a94251f0f333-tls-certs\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.170562 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.170431    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-home\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.170562 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.170454    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-644l9\" (UniqueName: \"kubernetes.io/projected/633b16f1-2a54-444a-be46-a94251f0f333-kube-api-access-644l9\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.170562 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.170479    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-tmp-dir\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.170562 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.170559    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-model-cache\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.170760 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.170586    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-dshm\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.170860 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.170836    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-home\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.170907 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.170854    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-tmp-dir\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.170956 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.170930    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-model-cache\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.172964 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.172933    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-dshm\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.173098 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.173082    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/633b16f1-2a54-444a-be46-a94251f0f333-tls-certs\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.178122 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.178105    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-644l9\" (UniqueName: \"kubernetes.io/projected/633b16f1-2a54-444a-be46-a94251f0f333-kube-api-access-644l9\") pod \"auth-test-model-e812a4-kserve-575c86cf95-kvv8d\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") " pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.273273 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.273189    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:09.403530 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.403466    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"]
Jul 10 02:49:09.405656 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:49:09.405628    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod633b16f1_2a54_444a_be46_a94251f0f333.slice/crio-c9ddbd37acfa20176e009808613a97bfb16b8d8951b7df83c830b0561a7b64d4 WatchSource:0}: Error finding container c9ddbd37acfa20176e009808613a97bfb16b8d8951b7df83c830b0561a7b64d4: Status 404 returned error can't find the container with id c9ddbd37acfa20176e009808613a97bfb16b8d8951b7df83c830b0561a7b64d4
Jul 10 02:49:09.822748 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.822713    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d" event={"ID":"633b16f1-2a54-444a-be46-a94251f0f333","Type":"ContainerStarted","Data":"143aa91e86a4756940a008f1e27966ce4aff1d98186b10c3cbec349c43120e75"}
Jul 10 02:49:09.822748 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.822754    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d" event={"ID":"633b16f1-2a54-444a-be46-a94251f0f333","Type":"ContainerStarted","Data":"c9ddbd37acfa20176e009808613a97bfb16b8d8951b7df83c830b0561a7b64d4"}
Jul 10 02:49:09.841044 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:09.840990    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d" podStartSLOduration=1.840973094 podStartE2EDuration="1.840973094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:49:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:49:09.839233545 +0000 UTC m=+2096.821991232" watchObservedRunningTime="2026-07-10 02:49:09.840973094 +0000 UTC m=+2096.823730781"
Jul 10 02:49:12.663925 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:12.663894    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:49:19.273410 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:19.273376    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:19.273909 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:19.273451    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:19.285637 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:19.285613    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:19.872153 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:19.872119    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:49:30.266133 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:30.266099    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:49:47.062407 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:49:47.062368    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:50:05.854263 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:50:05.854173    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:50:25.156128 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:50:25.156095    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:50:42.855117 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:50:42.855076    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:51:02.451533 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:02.451481    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:51:17.070375 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:17.070341    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"]
Jul 10 02:51:17.951638 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:17.951609    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"]
Jul 10 02:51:21.057851 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:21.057812    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:51:24.961481 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:24.961450    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"]
Jul 10 02:51:24.964489 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:24.964466    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:24.966925 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:24.966905    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e239ac\"/\"sub-test-model-e239ac-kserve-self-signed-certs\""
Jul 10 02:51:24.974258 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:24.974237    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"]
Jul 10 02:51:25.073846 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.073815    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-tls-certs\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.074032 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.073859    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-home\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.074032 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.073885    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-model-cache\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.074032 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.073968    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vxwd\" (UniqueName: \"kubernetes.io/projected/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-kube-api-access-8vxwd\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.074032 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.074012    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-tmp-dir\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.074208 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.074047    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-dshm\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.174634 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.174600    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-tls-certs\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.174634 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.174649    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-home\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.174866 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.174670    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-model-cache\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.174866 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.174696    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vxwd\" (UniqueName: \"kubernetes.io/projected/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-kube-api-access-8vxwd\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.174866 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.174724    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-tmp-dir\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.174866 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.174754    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-dshm\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.175055 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.175010    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-home\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.175111 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.175073    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-tmp-dir\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.175171 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.175147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-model-cache\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.177177 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.177154    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-dshm\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.177453 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.177433    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-tls-certs\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.182668 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.182653    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vxwd\" (UniqueName: \"kubernetes.io/projected/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-kube-api-access-8vxwd\") pod \"sub-test-model-e239ac-kserve-556579d469-wtxx2\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") " pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.273782 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.273700    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:25.400483 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:25.400455    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"]
Jul 10 02:51:25.403796 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:51:25.403774    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e547ce6_a3e1_48da_8abc_9b78d9f41aa9.slice/crio-82f15dfcd35d3479fba9d2f60fb2df6e3cf464a86d04aa8944c2d4b4d29ed929 WatchSource:0}: Error finding container 82f15dfcd35d3479fba9d2f60fb2df6e3cf464a86d04aa8944c2d4b4d29ed929: Status 404 returned error can't find the container with id 82f15dfcd35d3479fba9d2f60fb2df6e3cf464a86d04aa8944c2d4b4d29ed929
Jul 10 02:51:26.377960 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:26.377923    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2" event={"ID":"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9","Type":"ContainerStarted","Data":"eb560a5b4d665afae109c9fca53f7b25f88e0d255724d756ba6f6fa52eac27f4"}
Jul 10 02:51:26.377960 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:26.377965    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2" event={"ID":"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9","Type":"ContainerStarted","Data":"82f15dfcd35d3479fba9d2f60fb2df6e3cf464a86d04aa8944c2d4b4d29ed929"}
Jul 10 02:51:26.395966 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:26.395918    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2" podStartSLOduration=2.395902184 podStartE2EDuration="2.395902184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:51:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:51:26.394261647 +0000 UTC m=+2233.377019334" watchObservedRunningTime="2026-07-10 02:51:26.395902184 +0000 UTC m=+2233.378659870"
Jul 10 02:51:32.090165 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.090114    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj" podUID="4c961630-eb30-465c-86da-41c925cafca6" containerName="main" containerID="cri-o://99cfb56a240e769dc7410a2faa08aef4ec6c9b9c6af6ec6b7a7901e4d694a9c9" gracePeriod=45
Jul 10 02:51:32.376740 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.376717    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:51:32.406653 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.406614    2579 generic.go:356] "Generic (PLEG): container finished" podID="4c961630-eb30-465c-86da-41c925cafca6" containerID="99cfb56a240e769dc7410a2faa08aef4ec6c9b9c6af6ec6b7a7901e4d694a9c9" exitCode=0
Jul 10 02:51:32.406821 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.406652    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj" event={"ID":"4c961630-eb30-465c-86da-41c925cafca6","Type":"ContainerDied","Data":"99cfb56a240e769dc7410a2faa08aef4ec6c9b9c6af6ec6b7a7901e4d694a9c9"}
Jul 10 02:51:32.406821 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.406669    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"
Jul 10 02:51:32.406821 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.406703    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj" event={"ID":"4c961630-eb30-465c-86da-41c925cafca6","Type":"ContainerDied","Data":"c519fa92193c2c719ffe47382c6542e4babee4a9347e309945de9c0aac0dc581"}
Jul 10 02:51:32.406821 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.406726    2579 scope.go:117] "RemoveContainer" containerID="99cfb56a240e769dc7410a2faa08aef4ec6c9b9c6af6ec6b7a7901e4d694a9c9"
Jul 10 02:51:32.416137 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.416102    2579 scope.go:117] "RemoveContainer" containerID="99cfb56a240e769dc7410a2faa08aef4ec6c9b9c6af6ec6b7a7901e4d694a9c9"
Jul 10 02:51:32.416496 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:51:32.416469    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99cfb56a240e769dc7410a2faa08aef4ec6c9b9c6af6ec6b7a7901e4d694a9c9\": container with ID starting with 99cfb56a240e769dc7410a2faa08aef4ec6c9b9c6af6ec6b7a7901e4d694a9c9 not found: ID does not exist" containerID="99cfb56a240e769dc7410a2faa08aef4ec6c9b9c6af6ec6b7a7901e4d694a9c9"
Jul 10 02:51:32.416605 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.416534    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99cfb56a240e769dc7410a2faa08aef4ec6c9b9c6af6ec6b7a7901e4d694a9c9"} err="failed to get container status \"99cfb56a240e769dc7410a2faa08aef4ec6c9b9c6af6ec6b7a7901e4d694a9c9\": rpc error: code = NotFound desc = could not find container \"99cfb56a240e769dc7410a2faa08aef4ec6c9b9c6af6ec6b7a7901e4d694a9c9\": container with ID starting with 99cfb56a240e769dc7410a2faa08aef4ec6c9b9c6af6ec6b7a7901e4d694a9c9 not found: ID does not exist"
Jul 10 02:51:32.431818 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.431794    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-dshm\") pod \"4c961630-eb30-465c-86da-41c925cafca6\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") "
Jul 10 02:51:32.431932 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.431840    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwxmn\" (UniqueName: \"kubernetes.io/projected/4c961630-eb30-465c-86da-41c925cafca6-kube-api-access-nwxmn\") pod \"4c961630-eb30-465c-86da-41c925cafca6\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") "
Jul 10 02:51:32.431932 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.431875    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-model-cache\") pod \"4c961630-eb30-465c-86da-41c925cafca6\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") "
Jul 10 02:51:32.431932 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.431896    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c961630-eb30-465c-86da-41c925cafca6-tls-certs\") pod \"4c961630-eb30-465c-86da-41c925cafca6\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") "
Jul 10 02:51:32.431932 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.431914    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-tmp-dir\") pod \"4c961630-eb30-465c-86da-41c925cafca6\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") "
Jul 10 02:51:32.432495 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.431939    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-home\") pod \"4c961630-eb30-465c-86da-41c925cafca6\" (UID: \"4c961630-eb30-465c-86da-41c925cafca6\") "
Jul 10 02:51:32.432495 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.432111    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-model-cache" (OuterVolumeSpecName: "model-cache") pod "4c961630-eb30-465c-86da-41c925cafca6" (UID: "4c961630-eb30-465c-86da-41c925cafca6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:51:32.432495 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.432186    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4c961630-eb30-465c-86da-41c925cafca6" (UID: "4c961630-eb30-465c-86da-41c925cafca6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:51:32.432495 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.432227    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-model-cache\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:51:32.432495 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.432297    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-home" (OuterVolumeSpecName: "home") pod "4c961630-eb30-465c-86da-41c925cafca6" (UID: "4c961630-eb30-465c-86da-41c925cafca6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:51:32.434321 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.434291    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c961630-eb30-465c-86da-41c925cafca6-kube-api-access-nwxmn" (OuterVolumeSpecName: "kube-api-access-nwxmn") pod "4c961630-eb30-465c-86da-41c925cafca6" (UID: "4c961630-eb30-465c-86da-41c925cafca6"). InnerVolumeSpecName "kube-api-access-nwxmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:51:32.434565 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.434546    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-dshm" (OuterVolumeSpecName: "dshm") pod "4c961630-eb30-465c-86da-41c925cafca6" (UID: "4c961630-eb30-465c-86da-41c925cafca6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:51:32.434680 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.434662    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c961630-eb30-465c-86da-41c925cafca6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4c961630-eb30-465c-86da-41c925cafca6" (UID: "4c961630-eb30-465c-86da-41c925cafca6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:51:32.532929 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.532898    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-dshm\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:51:32.532929 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.532927    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwxmn\" (UniqueName: \"kubernetes.io/projected/4c961630-eb30-465c-86da-41c925cafca6-kube-api-access-nwxmn\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:51:32.533142 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.532937    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c961630-eb30-465c-86da-41c925cafca6-tls-certs\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:51:32.533142 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.532947    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-tmp-dir\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:51:32.533142 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.532954    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c961630-eb30-465c-86da-41c925cafca6-home\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:51:32.729330 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.729304    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"]
Jul 10 02:51:32.731213 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.731191    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/auth-test-model-e239ac-kserve-7994f94855-vwptj"]
Jul 10 02:51:32.971695 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:32.971604    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d" podUID="633b16f1-2a54-444a-be46-a94251f0f333" containerName="main" containerID="cri-o://143aa91e86a4756940a008f1e27966ce4aff1d98186b10c3cbec349c43120e75" gracePeriod=45
Jul 10 02:51:33.220728 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.220700    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:51:33.238301 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.238239    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-tmp-dir\") pod \"633b16f1-2a54-444a-be46-a94251f0f333\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") "
Jul 10 02:51:33.238301 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.238280    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-model-cache\") pod \"633b16f1-2a54-444a-be46-a94251f0f333\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") "
Jul 10 02:51:33.238478 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.238390    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/633b16f1-2a54-444a-be46-a94251f0f333-tls-certs\") pod \"633b16f1-2a54-444a-be46-a94251f0f333\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") "
Jul 10 02:51:33.239092 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.238592    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "633b16f1-2a54-444a-be46-a94251f0f333" (UID: "633b16f1-2a54-444a-be46-a94251f0f333"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:51:33.239092 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.238792    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-model-cache" (OuterVolumeSpecName: "model-cache") pod "633b16f1-2a54-444a-be46-a94251f0f333" (UID: "633b16f1-2a54-444a-be46-a94251f0f333"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:51:33.239283 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.239267    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-model-cache\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:51:33.239348 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.239287    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-tmp-dir\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:51:33.241121 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.241099    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/633b16f1-2a54-444a-be46-a94251f0f333-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "633b16f1-2a54-444a-be46-a94251f0f333" (UID: "633b16f1-2a54-444a-be46-a94251f0f333"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:51:33.340204 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.340178    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-dshm\") pod \"633b16f1-2a54-444a-be46-a94251f0f333\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") "
Jul 10 02:51:33.340452 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.340219    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-644l9\" (UniqueName: \"kubernetes.io/projected/633b16f1-2a54-444a-be46-a94251f0f333-kube-api-access-644l9\") pod \"633b16f1-2a54-444a-be46-a94251f0f333\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") "
Jul 10 02:51:33.340452 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.340248    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-home\") pod \"633b16f1-2a54-444a-be46-a94251f0f333\" (UID: \"633b16f1-2a54-444a-be46-a94251f0f333\") "
Jul 10 02:51:33.340605 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.340492    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-home" (OuterVolumeSpecName: "home") pod "633b16f1-2a54-444a-be46-a94251f0f333" (UID: "633b16f1-2a54-444a-be46-a94251f0f333"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:51:33.340605 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.340568    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/633b16f1-2a54-444a-be46-a94251f0f333-tls-certs\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:51:33.342385 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.342364    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-dshm" (OuterVolumeSpecName: "dshm") pod "633b16f1-2a54-444a-be46-a94251f0f333" (UID: "633b16f1-2a54-444a-be46-a94251f0f333"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:51:33.342484 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.342449    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/633b16f1-2a54-444a-be46-a94251f0f333-kube-api-access-644l9" (OuterVolumeSpecName: "kube-api-access-644l9") pod "633b16f1-2a54-444a-be46-a94251f0f333" (UID: "633b16f1-2a54-444a-be46-a94251f0f333"). InnerVolumeSpecName "kube-api-access-644l9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:51:33.411949 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.411917    2579 generic.go:356] "Generic (PLEG): container finished" podID="633b16f1-2a54-444a-be46-a94251f0f333" containerID="143aa91e86a4756940a008f1e27966ce4aff1d98186b10c3cbec349c43120e75" exitCode=0
Jul 10 02:51:33.412098 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.411947    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d" event={"ID":"633b16f1-2a54-444a-be46-a94251f0f333","Type":"ContainerDied","Data":"143aa91e86a4756940a008f1e27966ce4aff1d98186b10c3cbec349c43120e75"}
Jul 10 02:51:33.412098 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.411987    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d" event={"ID":"633b16f1-2a54-444a-be46-a94251f0f333","Type":"ContainerDied","Data":"c9ddbd37acfa20176e009808613a97bfb16b8d8951b7df83c830b0561a7b64d4"}
Jul 10 02:51:33.412098 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.412003    2579 scope.go:117] "RemoveContainer" containerID="143aa91e86a4756940a008f1e27966ce4aff1d98186b10c3cbec349c43120e75"
Jul 10 02:51:33.412098 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.411956    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"
Jul 10 02:51:33.420247 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.420231    2579 scope.go:117] "RemoveContainer" containerID="143aa91e86a4756940a008f1e27966ce4aff1d98186b10c3cbec349c43120e75"
Jul 10 02:51:33.420495 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:51:33.420475    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"143aa91e86a4756940a008f1e27966ce4aff1d98186b10c3cbec349c43120e75\": container with ID starting with 143aa91e86a4756940a008f1e27966ce4aff1d98186b10c3cbec349c43120e75 not found: ID does not exist" containerID="143aa91e86a4756940a008f1e27966ce4aff1d98186b10c3cbec349c43120e75"
Jul 10 02:51:33.420568 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.420523    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"143aa91e86a4756940a008f1e27966ce4aff1d98186b10c3cbec349c43120e75"} err="failed to get container status \"143aa91e86a4756940a008f1e27966ce4aff1d98186b10c3cbec349c43120e75\": rpc error: code = NotFound desc = could not find container \"143aa91e86a4756940a008f1e27966ce4aff1d98186b10c3cbec349c43120e75\": container with ID starting with 143aa91e86a4756940a008f1e27966ce4aff1d98186b10c3cbec349c43120e75 not found: ID does not exist"
Jul 10 02:51:33.432200 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.432166    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"]
Jul 10 02:51:33.435604 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.435585    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/auth-test-model-e812a4-kserve-575c86cf95-kvv8d"]
Jul 10 02:51:33.441708 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.441693    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-home\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:51:33.441761 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.441710    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/633b16f1-2a54-444a-be46-a94251f0f333-dshm\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:51:33.441761 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.441719    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-644l9\" (UniqueName: \"kubernetes.io/projected/633b16f1-2a54-444a-be46-a94251f0f333-kube-api-access-644l9\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:51:33.570716 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.570642    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c961630-eb30-465c-86da-41c925cafca6" path="/var/lib/kubelet/pods/4c961630-eb30-465c-86da-41c925cafca6/volumes"
Jul 10 02:51:33.571012 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:33.570999    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="633b16f1-2a54-444a-be46-a94251f0f333" path="/var/lib/kubelet/pods/633b16f1-2a54-444a-be46-a94251f0f333/volumes"
Jul 10 02:51:35.274545 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:35.274449    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:35.274545 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:35.274498    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:35.287097 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:35.287068    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:35.432884 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:35.432856    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:51:38.357775 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:51:38.357748    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:52:10.659580 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.659535    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"]
Jul 10 02:52:10.660133 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.659892    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="633b16f1-2a54-444a-be46-a94251f0f333" containerName="main"
Jul 10 02:52:10.660133 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.659906    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="633b16f1-2a54-444a-be46-a94251f0f333" containerName="main"
Jul 10 02:52:10.660133 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.659916    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c961630-eb30-465c-86da-41c925cafca6" containerName="main"
Jul 10 02:52:10.660133 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.659921    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c961630-eb30-465c-86da-41c925cafca6" containerName="main"
Jul 10 02:52:10.660133 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.659969    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="633b16f1-2a54-444a-be46-a94251f0f333" containerName="main"
Jul 10 02:52:10.660133 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.659977    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c961630-eb30-465c-86da-41c925cafca6" containerName="main"
Jul 10 02:52:10.663182 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.663154    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.665936 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.665910    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e812a4\"/\"kube-root-ca.crt\""
Jul 10 02:52:10.666989 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.666972    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e812a4\"/\"openshift-service-ca.crt\""
Jul 10 02:52:10.667191 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.666984    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e812a4\"/\"sub-test-model-e812a4-kserve-self-signed-certs\""
Jul 10 02:52:10.667546 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.667530    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e812a4\"/\"default-dockercfg-j8jr4\""
Jul 10 02:52:10.672208 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.672186    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"]
Jul 10 02:52:10.756077 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.756047    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9241114-9fe1-4bba-a153-c18078c75977-tls-certs\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.756239 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.756100    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6lbz\" (UniqueName: \"kubernetes.io/projected/c9241114-9fe1-4bba-a153-c18078c75977-kube-api-access-s6lbz\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.756239 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.756131    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-home\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.756239 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.756172    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-dshm\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.756239 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.756192    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-model-cache\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.756239 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.756219    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-tmp-dir\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.856771 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.856737    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-model-cache\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.856932 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.856781    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-tmp-dir\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.856932 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.856804    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9241114-9fe1-4bba-a153-c18078c75977-tls-certs\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.856932 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.856844    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6lbz\" (UniqueName: \"kubernetes.io/projected/c9241114-9fe1-4bba-a153-c18078c75977-kube-api-access-s6lbz\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.856932 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.856871    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-home\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.856932 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.856889    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-dshm\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.857217 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.857193    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-tmp-dir\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.857282 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.857220    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-model-cache\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.857282 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.857245    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-home\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.859368 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.859337    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-dshm\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.859482 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.859427    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9241114-9fe1-4bba-a153-c18078c75977-tls-certs\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.865085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.865064    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6lbz\" (UniqueName: \"kubernetes.io/projected/c9241114-9fe1-4bba-a153-c18078c75977-kube-api-access-s6lbz\") pod \"sub-test-model-e812a4-kserve-6578d5db97-vptz5\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") " pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:10.973061 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:10.972986    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:11.104552 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:11.104529    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"]
Jul 10 02:52:11.106471 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:52:11.106437    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9241114_9fe1_4bba_a153_c18078c75977.slice/crio-ddac709e616576a7c59ca4bfbd6a1efd5d46b5a297109ffc22effdedda03005d WatchSource:0}: Error finding container ddac709e616576a7c59ca4bfbd6a1efd5d46b5a297109ffc22effdedda03005d: Status 404 returned error can't find the container with id ddac709e616576a7c59ca4bfbd6a1efd5d46b5a297109ffc22effdedda03005d
Jul 10 02:52:11.571257 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:11.571216    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5" event={"ID":"c9241114-9fe1-4bba-a153-c18078c75977","Type":"ContainerStarted","Data":"a6c7cb93b70a32b917399efb8cc61f1412e7238d5702887df49447930d6ad4ab"}
Jul 10 02:52:11.571425 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:11.571265    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5" event={"ID":"c9241114-9fe1-4bba-a153-c18078c75977","Type":"ContainerStarted","Data":"ddac709e616576a7c59ca4bfbd6a1efd5d46b5a297109ffc22effdedda03005d"}
Jul 10 02:52:11.587754 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:11.587707    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5" podStartSLOduration=1.58769332 podStartE2EDuration="1.58769332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:52:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:52:11.586863795 +0000 UTC m=+2278.569621481" watchObservedRunningTime="2026-07-10 02:52:11.58769332 +0000 UTC m=+2278.570451005"
Jul 10 02:52:15.467133 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:15.467098    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:52:20.974088 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:20.974048    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:20.974544 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:20.974304    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:20.986534 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:20.986491    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:21.622287 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:21.622261    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:52:33.758072 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:33.758039    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:52:48.160257 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:48.160228    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:52:53.692657 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:53.692625    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"]
Jul 10 02:52:54.565626 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:52:54.565587    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"]
Jul 10 02:53:08.152278 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:08.152198    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:53:08.712658 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:08.712593    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2" podUID="1e547ce6-a3e1-48da-8abc-9b78d9f41aa9" containerName="main" containerID="cri-o://eb560a5b4d665afae109c9fca53f7b25f88e0d255724d756ba6f6fa52eac27f4" gracePeriod=45
Jul 10 02:53:08.962549 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:08.962526    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:53:09.068330 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.068234    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-home\") pod \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") "
Jul 10 02:53:09.068330 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.068275    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-tmp-dir\") pod \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") "
Jul 10 02:53:09.068330 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.068315    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-model-cache\") pod \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") "
Jul 10 02:53:09.068650 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.068365    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-tls-certs\") pod \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") "
Jul 10 02:53:09.068650 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.068420    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vxwd\" (UniqueName: \"kubernetes.io/projected/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-kube-api-access-8vxwd\") pod \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") "
Jul 10 02:53:09.068650 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.068460    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-dshm\") pod \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\" (UID: \"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9\") "
Jul 10 02:53:09.068650 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.068605    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1e547ce6-a3e1-48da-8abc-9b78d9f41aa9" (UID: "1e547ce6-a3e1-48da-8abc-9b78d9f41aa9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:53:09.068650 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.068622    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-home" (OuterVolumeSpecName: "home") pod "1e547ce6-a3e1-48da-8abc-9b78d9f41aa9" (UID: "1e547ce6-a3e1-48da-8abc-9b78d9f41aa9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:53:09.068650 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.068633    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-model-cache" (OuterVolumeSpecName: "model-cache") pod "1e547ce6-a3e1-48da-8abc-9b78d9f41aa9" (UID: "1e547ce6-a3e1-48da-8abc-9b78d9f41aa9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:53:09.068904 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.068773    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-tmp-dir\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:53:09.068904 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.068790    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-model-cache\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:53:09.068904 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.068805    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-home\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:53:09.070782 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.070754    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1e547ce6-a3e1-48da-8abc-9b78d9f41aa9" (UID: "1e547ce6-a3e1-48da-8abc-9b78d9f41aa9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:53:09.070859 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.070802    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-kube-api-access-8vxwd" (OuterVolumeSpecName: "kube-api-access-8vxwd") pod "1e547ce6-a3e1-48da-8abc-9b78d9f41aa9" (UID: "1e547ce6-a3e1-48da-8abc-9b78d9f41aa9"). InnerVolumeSpecName "kube-api-access-8vxwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:53:09.070900 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.070828    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-dshm" (OuterVolumeSpecName: "dshm") pod "1e547ce6-a3e1-48da-8abc-9b78d9f41aa9" (UID: "1e547ce6-a3e1-48da-8abc-9b78d9f41aa9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:53:09.169414 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.169374    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-tls-certs\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:53:09.169414 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.169409    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vxwd\" (UniqueName: \"kubernetes.io/projected/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-kube-api-access-8vxwd\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:53:09.169414 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.169418    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9-dshm\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:53:09.585737 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.585698    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5" podUID="c9241114-9fe1-4bba-a153-c18078c75977" containerName="main" containerID="cri-o://a6c7cb93b70a32b917399efb8cc61f1412e7238d5702887df49447930d6ad4ab" gracePeriod=45
Jul 10 02:53:09.811231 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.811199    2579 generic.go:356] "Generic (PLEG): container finished" podID="1e547ce6-a3e1-48da-8abc-9b78d9f41aa9" containerID="eb560a5b4d665afae109c9fca53f7b25f88e0d255724d756ba6f6fa52eac27f4" exitCode=0
Jul 10 02:53:09.811409 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.811244    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"
Jul 10 02:53:09.811409 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.811274    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2" event={"ID":"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9","Type":"ContainerDied","Data":"eb560a5b4d665afae109c9fca53f7b25f88e0d255724d756ba6f6fa52eac27f4"}
Jul 10 02:53:09.811409 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.811314    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2" event={"ID":"1e547ce6-a3e1-48da-8abc-9b78d9f41aa9","Type":"ContainerDied","Data":"82f15dfcd35d3479fba9d2f60fb2df6e3cf464a86d04aa8944c2d4b4d29ed929"}
Jul 10 02:53:09.811409 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.811339    2579 scope.go:117] "RemoveContainer" containerID="eb560a5b4d665afae109c9fca53f7b25f88e0d255724d756ba6f6fa52eac27f4"
Jul 10 02:53:09.813237 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.813213    2579 generic.go:356] "Generic (PLEG): container finished" podID="c9241114-9fe1-4bba-a153-c18078c75977" containerID="a6c7cb93b70a32b917399efb8cc61f1412e7238d5702887df49447930d6ad4ab" exitCode=0
Jul 10 02:53:09.813367 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.813296    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5" event={"ID":"c9241114-9fe1-4bba-a153-c18078c75977","Type":"ContainerDied","Data":"a6c7cb93b70a32b917399efb8cc61f1412e7238d5702887df49447930d6ad4ab"}
Jul 10 02:53:09.820784 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.820767    2579 scope.go:117] "RemoveContainer" containerID="eb560a5b4d665afae109c9fca53f7b25f88e0d255724d756ba6f6fa52eac27f4"
Jul 10 02:53:09.821079 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:53:09.821050    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb560a5b4d665afae109c9fca53f7b25f88e0d255724d756ba6f6fa52eac27f4\": container with ID starting with eb560a5b4d665afae109c9fca53f7b25f88e0d255724d756ba6f6fa52eac27f4 not found: ID does not exist" containerID="eb560a5b4d665afae109c9fca53f7b25f88e0d255724d756ba6f6fa52eac27f4"
Jul 10 02:53:09.821154 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.821090    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb560a5b4d665afae109c9fca53f7b25f88e0d255724d756ba6f6fa52eac27f4"} err="failed to get container status \"eb560a5b4d665afae109c9fca53f7b25f88e0d255724d756ba6f6fa52eac27f4\": rpc error: code = NotFound desc = could not find container \"eb560a5b4d665afae109c9fca53f7b25f88e0d255724d756ba6f6fa52eac27f4\": container with ID starting with eb560a5b4d665afae109c9fca53f7b25f88e0d255724d756ba6f6fa52eac27f4 not found: ID does not exist"
Jul 10 02:53:09.829605 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.829582    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"]
Jul 10 02:53:09.833673 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.833654    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/sub-test-model-e239ac-kserve-556579d469-wtxx2"]
Jul 10 02:53:09.844009 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.843994    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:53:09.975947 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.975917    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-home\") pod \"c9241114-9fe1-4bba-a153-c18078c75977\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") "
Jul 10 02:53:09.975947 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.975953    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-tmp-dir\") pod \"c9241114-9fe1-4bba-a153-c18078c75977\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") "
Jul 10 02:53:09.976198 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.975984    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-dshm\") pod \"c9241114-9fe1-4bba-a153-c18078c75977\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") "
Jul 10 02:53:09.976198 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.975998    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-model-cache\") pod \"c9241114-9fe1-4bba-a153-c18078c75977\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") "
Jul 10 02:53:09.976198 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.976022    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9241114-9fe1-4bba-a153-c18078c75977-tls-certs\") pod \"c9241114-9fe1-4bba-a153-c18078c75977\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") "
Jul 10 02:53:09.976198 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.976081    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6lbz\" (UniqueName: \"kubernetes.io/projected/c9241114-9fe1-4bba-a153-c18078c75977-kube-api-access-s6lbz\") pod \"c9241114-9fe1-4bba-a153-c18078c75977\" (UID: \"c9241114-9fe1-4bba-a153-c18078c75977\") "
Jul 10 02:53:09.976381 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.976242    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-home" (OuterVolumeSpecName: "home") pod "c9241114-9fe1-4bba-a153-c18078c75977" (UID: "c9241114-9fe1-4bba-a153-c18078c75977"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:53:09.976381 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.976283    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-model-cache" (OuterVolumeSpecName: "model-cache") pod "c9241114-9fe1-4bba-a153-c18078c75977" (UID: "c9241114-9fe1-4bba-a153-c18078c75977"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:53:09.976381 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.976298    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c9241114-9fe1-4bba-a153-c18078c75977" (UID: "c9241114-9fe1-4bba-a153-c18078c75977"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:53:09.978278 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.978248    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-dshm" (OuterVolumeSpecName: "dshm") pod "c9241114-9fe1-4bba-a153-c18078c75977" (UID: "c9241114-9fe1-4bba-a153-c18078c75977"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:53:09.978388 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.978309    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9241114-9fe1-4bba-a153-c18078c75977-kube-api-access-s6lbz" (OuterVolumeSpecName: "kube-api-access-s6lbz") pod "c9241114-9fe1-4bba-a153-c18078c75977" (UID: "c9241114-9fe1-4bba-a153-c18078c75977"). InnerVolumeSpecName "kube-api-access-s6lbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:53:09.978388 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:09.978325    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9241114-9fe1-4bba-a153-c18078c75977-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c9241114-9fe1-4bba-a153-c18078c75977" (UID: "c9241114-9fe1-4bba-a153-c18078c75977"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:53:10.076871 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:10.076789    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-home\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:53:10.076871 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:10.076833    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-tmp-dir\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:53:10.076871 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:10.076842    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-dshm\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:53:10.076871 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:10.076850    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9241114-9fe1-4bba-a153-c18078c75977-model-cache\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:53:10.076871 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:10.076859    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9241114-9fe1-4bba-a153-c18078c75977-tls-certs\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:53:10.076871 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:10.076868    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6lbz\" (UniqueName: \"kubernetes.io/projected/c9241114-9fe1-4bba-a153-c18078c75977-kube-api-access-s6lbz\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:53:10.819231 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:10.819199    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5" event={"ID":"c9241114-9fe1-4bba-a153-c18078c75977","Type":"ContainerDied","Data":"ddac709e616576a7c59ca4bfbd6a1efd5d46b5a297109ffc22effdedda03005d"}
Jul 10 02:53:10.819678 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:10.819245    2579 scope.go:117] "RemoveContainer" containerID="a6c7cb93b70a32b917399efb8cc61f1412e7238d5702887df49447930d6ad4ab"
Jul 10 02:53:10.819678 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:10.819206    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"
Jul 10 02:53:10.839140 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:10.839117    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"]
Jul 10 02:53:10.843074 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:10.843055    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/sub-test-model-e812a4-kserve-6578d5db97-vptz5"]
Jul 10 02:53:11.565921 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:11.565888    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e547ce6-a3e1-48da-8abc-9b78d9f41aa9" path="/var/lib/kubelet/pods/1e547ce6-a3e1-48da-8abc-9b78d9f41aa9/volumes"
Jul 10 02:53:11.566259 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:11.566246    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9241114-9fe1-4bba-a153-c18078c75977" path="/var/lib/kubelet/pods/c9241114-9fe1-4bba-a153-c18078c75977/volumes"
Jul 10 02:53:34.259358 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.259329    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"]
Jul 10 02:53:34.259771 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.259619    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e547ce6-a3e1-48da-8abc-9b78d9f41aa9" containerName="main"
Jul 10 02:53:34.259771 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.259629    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e547ce6-a3e1-48da-8abc-9b78d9f41aa9" containerName="main"
Jul 10 02:53:34.259771 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.259642    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9241114-9fe1-4bba-a153-c18078c75977" containerName="main"
Jul 10 02:53:34.259771 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.259648    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9241114-9fe1-4bba-a153-c18078c75977" containerName="main"
Jul 10 02:53:34.259771 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.259704    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e547ce6-a3e1-48da-8abc-9b78d9f41aa9" containerName="main"
Jul 10 02:53:34.259771 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.259712    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9241114-9fe1-4bba-a153-c18078c75977" containerName="main"
Jul 10 02:53:34.262550 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.262534    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.265083 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.265062    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e239ac\"/\"kube-root-ca.crt\""
Jul 10 02:53:34.266234 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.266216    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e239ac\"/\"default-dockercfg-5hv2n\""
Jul 10 02:53:34.266339 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.266249    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e239ac\"/\"openshift-service-ca.crt\""
Jul 10 02:53:34.266339 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.266250    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e239ac\"/\"rate-test-model-e239ac-kserve-self-signed-certs\""
Jul 10 02:53:34.276251 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.276216    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"]
Jul 10 02:53:34.355258 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.355226    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-tls-certs\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.355429 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.355264    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-home\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.355429 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.355304    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-model-cache\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.355429 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.355327    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-tmp-dir\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.355429 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.355352    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgszp\" (UniqueName: \"kubernetes.io/projected/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-kube-api-access-zgszp\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.355647 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.355448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-dshm\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.456745 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.456707    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-dshm\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.456946 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.456776    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-tls-certs\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.456946 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.456807    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-home\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.456946 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.456843    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-model-cache\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.456946 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.456868    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-tmp-dir\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.456946 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.456908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgszp\" (UniqueName: \"kubernetes.io/projected/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-kube-api-access-zgszp\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.457232 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.457204    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-home\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.457291 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.457255    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-model-cache\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.457329 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.457264    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-tmp-dir\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.459168 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.459147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-dshm\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.459480 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.459462    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-tls-certs\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.466548 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.466524    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgszp\" (UniqueName: \"kubernetes.io/projected/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-kube-api-access-zgszp\") pod \"rate-test-model-e239ac-kserve-ff7dc968c-n5pmz\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") " pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.571837 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.571761    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:34.707017 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.706992    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"]
Jul 10 02:53:34.709416 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:53:34.709389    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a98b5ab_675b_4c8e_baab_b6e2b02cfc87.slice/crio-952d09b9232b9290fddbfd53d852bff3ce426720d97653e800e4912f7fad44db WatchSource:0}: Error finding container 952d09b9232b9290fddbfd53d852bff3ce426720d97653e800e4912f7fad44db: Status 404 returned error can't find the container with id 952d09b9232b9290fddbfd53d852bff3ce426720d97653e800e4912f7fad44db
Jul 10 02:53:34.915762 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.915717    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz" event={"ID":"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87","Type":"ContainerStarted","Data":"f85b074a180935879d72605784954eafdb9b25af19ab6d6263818ab9e950412e"}
Jul 10 02:53:34.915762 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.915767    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz" event={"ID":"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87","Type":"ContainerStarted","Data":"952d09b9232b9290fddbfd53d852bff3ce426720d97653e800e4912f7fad44db"}
Jul 10 02:53:34.933831 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:34.933785    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz" podStartSLOduration=0.933769488 podStartE2EDuration="933.769488ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:53:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:53:34.931859191 +0000 UTC m=+2361.914616877" watchObservedRunningTime="2026-07-10 02:53:34.933769488 +0000 UTC m=+2361.916527167"
Jul 10 02:53:38.464457 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:38.464405    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:53:44.572886 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:44.572841    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:44.573278 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:44.572898    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:44.586145 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:44.586116    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:53:44.967764 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:53:44.967735    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:54:15.047812 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.047779    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"]
Jul 10 02:54:15.051272 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.051252    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.054058 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.054036    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e812a4\"/\"default-dockercfg-j8jr4\""
Jul 10 02:54:15.054172 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.054152    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e812a4\"/\"openshift-service-ca.crt\""
Jul 10 02:54:15.055176 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.055158    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e812a4\"/\"rate-test-model-e812a4-kserve-self-signed-certs\""
Jul 10 02:54:15.055260 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.055178    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e812a4\"/\"kube-root-ca.crt\""
Jul 10 02:54:15.062113 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.062093    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"]
Jul 10 02:54:15.092983 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.092959    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-tls-certs\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.093112 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.092997    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-model-cache\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.093112 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.093023    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-tmp-dir\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.093112 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.093081    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-home\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.093213 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.093118    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs9dx\" (UniqueName: \"kubernetes.io/projected/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-kube-api-access-bs9dx\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.093213 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.093148    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-dshm\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.193464 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.193434    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-model-cache\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.193629 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.193474    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-tmp-dir\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.193671 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.193664    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-home\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.193715 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.193687    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs9dx\" (UniqueName: \"kubernetes.io/projected/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-kube-api-access-bs9dx\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.193715 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.193712    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-dshm\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.193813 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.193734    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-tls-certs\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.193862 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.193835    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-model-cache\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.193914 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.193894    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-tmp-dir\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.193965 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.193943    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-home\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.195984 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.195962    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-dshm\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.196347 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.196329    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-tls-certs\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.202251 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.202231    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs9dx\" (UniqueName: \"kubernetes.io/projected/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-kube-api-access-bs9dx\") pod \"rate-test-model-e812a4-kserve-596ccc68bc-n87nb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") " pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.360640 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.360605    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:15.489977 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:15.489949    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"]
Jul 10 02:54:15.491962 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:54:15.491934    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fd49b12_988a_4eb5_b019_1f829f6b7bbb.slice/crio-31f13bffaca2ff4c425427241377b75920300bde7b013a995d085c0d45c1147d WatchSource:0}: Error finding container 31f13bffaca2ff4c425427241377b75920300bde7b013a995d085c0d45c1147d: Status 404 returned error can't find the container with id 31f13bffaca2ff4c425427241377b75920300bde7b013a995d085c0d45c1147d
Jul 10 02:54:16.083634 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:16.083599    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb" event={"ID":"3fd49b12-988a-4eb5-b019-1f829f6b7bbb","Type":"ContainerStarted","Data":"e8088e180f50b37da420be67b1e5a6471e4d5205062f31bb5e7e61a84b982299"}
Jul 10 02:54:16.084024 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:16.083651    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb" event={"ID":"3fd49b12-988a-4eb5-b019-1f829f6b7bbb","Type":"ContainerStarted","Data":"31f13bffaca2ff4c425427241377b75920300bde7b013a995d085c0d45c1147d"}
Jul 10 02:54:16.104300 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:16.104258    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb" podStartSLOduration=1.104243436 podStartE2EDuration="1.104243436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:54:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:54:16.103171833 +0000 UTC m=+2403.085929541" watchObservedRunningTime="2026-07-10 02:54:16.104243436 +0000 UTC m=+2403.087001121"
Jul 10 02:54:19.662071 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:19.662035    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:54:25.360769 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:25.360731    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:25.361229 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:25.360785    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:25.373040 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:25.373023    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:26.133687 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:26.133661    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:54:39.187207 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:39.187135    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:54:56.558374 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:54:56.558331    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:55:04.512982 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:04.512941    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"]
Jul 10 02:55:05.470959 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:05.470928    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"]
Jul 10 02:55:10.866641 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:10.866609    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:55:19.533278 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.533239    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz" podUID="0a98b5ab-675b-4c8e-baab-b6e2b02cfc87" containerName="main" containerID="cri-o://f85b074a180935879d72605784954eafdb9b25af19ab6d6263818ab9e950412e" gracePeriod=45
Jul 10 02:55:19.787844 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.787787    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:55:19.822276 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.822250    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-model-cache\") pod \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") "
Jul 10 02:55:19.822436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.822294    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-tls-certs\") pod \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") "
Jul 10 02:55:19.822436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.822328    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgszp\" (UniqueName: \"kubernetes.io/projected/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-kube-api-access-zgszp\") pod \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") "
Jul 10 02:55:19.822436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.822375    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-tmp-dir\") pod \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") "
Jul 10 02:55:19.822436 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.822400    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-dshm\") pod \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") "
Jul 10 02:55:19.822689 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.822549    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-home\") pod \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\" (UID: \"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87\") "
Jul 10 02:55:19.822689 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.822562    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-model-cache" (OuterVolumeSpecName: "model-cache") pod "0a98b5ab-675b-4c8e-baab-b6e2b02cfc87" (UID: "0a98b5ab-675b-4c8e-baab-b6e2b02cfc87"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:19.822689 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.822636    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0a98b5ab-675b-4c8e-baab-b6e2b02cfc87" (UID: "0a98b5ab-675b-4c8e-baab-b6e2b02cfc87"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:19.822864 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.822839    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-home" (OuterVolumeSpecName: "home") pod "0a98b5ab-675b-4c8e-baab-b6e2b02cfc87" (UID: "0a98b5ab-675b-4c8e-baab-b6e2b02cfc87"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:19.822864 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.822858    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-tmp-dir\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:19.822989 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.822876    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-model-cache\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:19.824663 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.824640    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0a98b5ab-675b-4c8e-baab-b6e2b02cfc87" (UID: "0a98b5ab-675b-4c8e-baab-b6e2b02cfc87"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:55:19.824663 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.824647    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-dshm" (OuterVolumeSpecName: "dshm") pod "0a98b5ab-675b-4c8e-baab-b6e2b02cfc87" (UID: "0a98b5ab-675b-4c8e-baab-b6e2b02cfc87"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:19.824790 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.824727    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-kube-api-access-zgszp" (OuterVolumeSpecName: "kube-api-access-zgszp") pod "0a98b5ab-675b-4c8e-baab-b6e2b02cfc87" (UID: "0a98b5ab-675b-4c8e-baab-b6e2b02cfc87"). InnerVolumeSpecName "kube-api-access-zgszp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:55:19.923570 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.923537    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-home\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:19.923570 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.923566    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-tls-certs\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:19.923570 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.923577    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgszp\" (UniqueName: \"kubernetes.io/projected/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-kube-api-access-zgszp\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:19.923789 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:19.923587    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87-dshm\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:20.349256 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.349224    2579 generic.go:356] "Generic (PLEG): container finished" podID="0a98b5ab-675b-4c8e-baab-b6e2b02cfc87" containerID="f85b074a180935879d72605784954eafdb9b25af19ab6d6263818ab9e950412e" exitCode=0
Jul 10 02:55:20.349435 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.349268    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"
Jul 10 02:55:20.349435 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.349306    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz" event={"ID":"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87","Type":"ContainerDied","Data":"f85b074a180935879d72605784954eafdb9b25af19ab6d6263818ab9e950412e"}
Jul 10 02:55:20.349435 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.349353    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz" event={"ID":"0a98b5ab-675b-4c8e-baab-b6e2b02cfc87","Type":"ContainerDied","Data":"952d09b9232b9290fddbfd53d852bff3ce426720d97653e800e4912f7fad44db"}
Jul 10 02:55:20.349435 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.349375    2579 scope.go:117] "RemoveContainer" containerID="f85b074a180935879d72605784954eafdb9b25af19ab6d6263818ab9e950412e"
Jul 10 02:55:20.358148 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.358128    2579 scope.go:117] "RemoveContainer" containerID="f85b074a180935879d72605784954eafdb9b25af19ab6d6263818ab9e950412e"
Jul 10 02:55:20.358395 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:55:20.358376    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f85b074a180935879d72605784954eafdb9b25af19ab6d6263818ab9e950412e\": container with ID starting with f85b074a180935879d72605784954eafdb9b25af19ab6d6263818ab9e950412e not found: ID does not exist" containerID="f85b074a180935879d72605784954eafdb9b25af19ab6d6263818ab9e950412e"
Jul 10 02:55:20.358447 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.358405    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f85b074a180935879d72605784954eafdb9b25af19ab6d6263818ab9e950412e"} err="failed to get container status \"f85b074a180935879d72605784954eafdb9b25af19ab6d6263818ab9e950412e\": rpc error: code = NotFound desc = could not find container \"f85b074a180935879d72605784954eafdb9b25af19ab6d6263818ab9e950412e\": container with ID starting with f85b074a180935879d72605784954eafdb9b25af19ab6d6263818ab9e950412e not found: ID does not exist"
Jul 10 02:55:20.370619 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.370596    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"]
Jul 10 02:55:20.373991 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.373970    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e239ac/rate-test-model-e239ac-kserve-ff7dc968c-n5pmz"]
Jul 10 02:55:20.490894 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.490835    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb" podUID="3fd49b12-988a-4eb5-b019-1f829f6b7bbb" containerName="main" containerID="cri-o://e8088e180f50b37da420be67b1e5a6471e4d5205062f31bb5e7e61a84b982299" gracePeriod=45
Jul 10 02:55:20.743657 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.743633    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:55:20.830466 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.830431    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-model-cache\") pod \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") "
Jul 10 02:55:20.830678 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.830476    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-dshm\") pod \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") "
Jul 10 02:55:20.830678 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.830529    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-tmp-dir\") pod \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") "
Jul 10 02:55:20.830778 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.830722    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-model-cache" (OuterVolumeSpecName: "model-cache") pod "3fd49b12-988a-4eb5-b019-1f829f6b7bbb" (UID: "3fd49b12-988a-4eb5-b019-1f829f6b7bbb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:20.830856 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.830833    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3fd49b12-988a-4eb5-b019-1f829f6b7bbb" (UID: "3fd49b12-988a-4eb5-b019-1f829f6b7bbb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:20.832591 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.832572    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-dshm" (OuterVolumeSpecName: "dshm") pod "3fd49b12-988a-4eb5-b019-1f829f6b7bbb" (UID: "3fd49b12-988a-4eb5-b019-1f829f6b7bbb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:20.931799 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.931708    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-tls-certs\") pod \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") "
Jul 10 02:55:20.931799 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.931756    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-home\") pod \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") "
Jul 10 02:55:20.931799 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.931779    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs9dx\" (UniqueName: \"kubernetes.io/projected/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-kube-api-access-bs9dx\") pod \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\" (UID: \"3fd49b12-988a-4eb5-b019-1f829f6b7bbb\") "
Jul 10 02:55:20.932090 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.931918    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-model-cache\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:20.932090 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.931927    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-dshm\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:20.932090 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.931935    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-tmp-dir\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:20.932090 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.932012    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-home" (OuterVolumeSpecName: "home") pod "3fd49b12-988a-4eb5-b019-1f829f6b7bbb" (UID: "3fd49b12-988a-4eb5-b019-1f829f6b7bbb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:55:20.933820 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.933798    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3fd49b12-988a-4eb5-b019-1f829f6b7bbb" (UID: "3fd49b12-988a-4eb5-b019-1f829f6b7bbb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:55:20.933956 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:20.933934    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-kube-api-access-bs9dx" (OuterVolumeSpecName: "kube-api-access-bs9dx") pod "3fd49b12-988a-4eb5-b019-1f829f6b7bbb" (UID: "3fd49b12-988a-4eb5-b019-1f829f6b7bbb"). InnerVolumeSpecName "kube-api-access-bs9dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:55:21.032587 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.032553    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-tls-certs\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:21.032587 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.032583    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-home\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:21.032587 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.032594    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs9dx\" (UniqueName: \"kubernetes.io/projected/3fd49b12-988a-4eb5-b019-1f829f6b7bbb-kube-api-access-bs9dx\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:21.355608 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.355574    2579 generic.go:356] "Generic (PLEG): container finished" podID="3fd49b12-988a-4eb5-b019-1f829f6b7bbb" containerID="e8088e180f50b37da420be67b1e5a6471e4d5205062f31bb5e7e61a84b982299" exitCode=0
Jul 10 02:55:21.355801 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.355622    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb" event={"ID":"3fd49b12-988a-4eb5-b019-1f829f6b7bbb","Type":"ContainerDied","Data":"e8088e180f50b37da420be67b1e5a6471e4d5205062f31bb5e7e61a84b982299"}
Jul 10 02:55:21.355801 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.355640    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"
Jul 10 02:55:21.355801 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.355668    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb" event={"ID":"3fd49b12-988a-4eb5-b019-1f829f6b7bbb","Type":"ContainerDied","Data":"31f13bffaca2ff4c425427241377b75920300bde7b013a995d085c0d45c1147d"}
Jul 10 02:55:21.355801 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.355686    2579 scope.go:117] "RemoveContainer" containerID="e8088e180f50b37da420be67b1e5a6471e4d5205062f31bb5e7e61a84b982299"
Jul 10 02:55:21.364280 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.364266    2579 scope.go:117] "RemoveContainer" containerID="e8088e180f50b37da420be67b1e5a6471e4d5205062f31bb5e7e61a84b982299"
Jul 10 02:55:21.364570 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:55:21.364497    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8088e180f50b37da420be67b1e5a6471e4d5205062f31bb5e7e61a84b982299\": container with ID starting with e8088e180f50b37da420be67b1e5a6471e4d5205062f31bb5e7e61a84b982299 not found: ID does not exist" containerID="e8088e180f50b37da420be67b1e5a6471e4d5205062f31bb5e7e61a84b982299"
Jul 10 02:55:21.364623 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.364576    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8088e180f50b37da420be67b1e5a6471e4d5205062f31bb5e7e61a84b982299"} err="failed to get container status \"e8088e180f50b37da420be67b1e5a6471e4d5205062f31bb5e7e61a84b982299\": rpc error: code = NotFound desc = could not find container \"e8088e180f50b37da420be67b1e5a6471e4d5205062f31bb5e7e61a84b982299\": container with ID starting with e8088e180f50b37da420be67b1e5a6471e4d5205062f31bb5e7e61a84b982299 not found: ID does not exist"
Jul 10 02:55:21.377212 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.377190    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"]
Jul 10 02:55:21.380534 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.380488    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e812a4/rate-test-model-e812a4-kserve-596ccc68bc-n87nb"]
Jul 10 02:55:21.565731 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.565701    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a98b5ab-675b-4c8e-baab-b6e2b02cfc87" path="/var/lib/kubelet/pods/0a98b5ab-675b-4c8e-baab-b6e2b02cfc87/volumes"
Jul 10 02:55:21.566080 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.566067    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fd49b12-988a-4eb5-b019-1f829f6b7bbb" path="/var/lib/kubelet/pods/3fd49b12-988a-4eb5-b019-1f829f6b7bbb/volumes"
Jul 10 02:55:21.658116 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.658047    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7523-140119920548720"]
Jul 10 02:55:21.658327 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.658316    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fd49b12-988a-4eb5-b019-1f829f6b7bbb" containerName="main"
Jul 10 02:55:21.658369 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.658328    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fd49b12-988a-4eb5-b019-1f829f6b7bbb" containerName="main"
Jul 10 02:55:21.658369 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.658344    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a98b5ab-675b-4c8e-baab-b6e2b02cfc87" containerName="main"
Jul 10 02:55:21.658369 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.658350    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a98b5ab-675b-4c8e-baab-b6e2b02cfc87" containerName="main"
Jul 10 02:55:21.658468 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.658400    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a98b5ab-675b-4c8e-baab-b6e2b02cfc87" containerName="main"
Jul 10 02:55:21.658468 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.658410    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fd49b12-988a-4eb5-b019-1f829f6b7bbb" containerName="main"
Jul 10 02:55:21.662481 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.662458    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7523-140119920548720"
Jul 10 02:55:21.665252 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.665236    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-zzmwp\""
Jul 10 02:55:21.669693 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.669527    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7523-140119920548720"]
Jul 10 02:55:21.737720 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.737680    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xphm5\" (UniqueName: \"kubernetes.io/projected/79438ef9-9241-43d0-9808-bde56f9b472e-kube-api-access-xphm5\") pod \"test-curl-7523-140119920548720\" (UID: \"79438ef9-9241-43d0-9808-bde56f9b472e\") " pod="openshift-ingress/test-curl-7523-140119920548720"
Jul 10 02:55:21.838288 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.838250    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xphm5\" (UniqueName: \"kubernetes.io/projected/79438ef9-9241-43d0-9808-bde56f9b472e-kube-api-access-xphm5\") pod \"test-curl-7523-140119920548720\" (UID: \"79438ef9-9241-43d0-9808-bde56f9b472e\") " pod="openshift-ingress/test-curl-7523-140119920548720"
Jul 10 02:55:21.846120 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.846094    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xphm5\" (UniqueName: \"kubernetes.io/projected/79438ef9-9241-43d0-9808-bde56f9b472e-kube-api-access-xphm5\") pod \"test-curl-7523-140119920548720\" (UID: \"79438ef9-9241-43d0-9808-bde56f9b472e\") " pod="openshift-ingress/test-curl-7523-140119920548720"
Jul 10 02:55:21.972662 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:21.972572    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7523-140119920548720"
Jul 10 02:55:22.099553 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:22.099528    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7523-140119920548720"]
Jul 10 02:55:22.102111 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:55:22.102082    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79438ef9_9241_43d0_9808_bde56f9b472e.slice/crio-118dc1153ace92820658aaefbb737a36a05f51d97c635de7bb55eb3aa5a47e26 WatchSource:0}: Error finding container 118dc1153ace92820658aaefbb737a36a05f51d97c635de7bb55eb3aa5a47e26: Status 404 returned error can't find the container with id 118dc1153ace92820658aaefbb737a36a05f51d97c635de7bb55eb3aa5a47e26
Jul 10 02:55:22.103220 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:22.103197    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:55:22.361187 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:22.361155    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7523-140119920548720" event={"ID":"79438ef9-9241-43d0-9808-bde56f9b472e","Type":"ContainerStarted","Data":"118dc1153ace92820658aaefbb737a36a05f51d97c635de7bb55eb3aa5a47e26"}
Jul 10 02:55:23.367698 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:23.367663    2579 generic.go:356] "Generic (PLEG): container finished" podID="79438ef9-9241-43d0-9808-bde56f9b472e" containerID="e4715e0ed0c09c6cc220c96b18799a8377df31fb92dc2d615136edcb4e398338" exitCode=0
Jul 10 02:55:23.368085 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:23.367748    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7523-140119920548720" event={"ID":"79438ef9-9241-43d0-9808-bde56f9b472e","Type":"ContainerDied","Data":"e4715e0ed0c09c6cc220c96b18799a8377df31fb92dc2d615136edcb4e398338"}
Jul 10 02:55:24.506055 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:24.506032    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7523-140119920548720"
Jul 10 02:55:24.541656 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:24.541633    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7523-140119920548720_79438ef9-9241-43d0-9808-bde56f9b472e/test-curl-7523-140119920548720/0.log"
Jul 10 02:55:24.561194 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:24.561165    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xphm5\" (UniqueName: \"kubernetes.io/projected/79438ef9-9241-43d0-9808-bde56f9b472e-kube-api-access-xphm5\") pod \"79438ef9-9241-43d0-9808-bde56f9b472e\" (UID: \"79438ef9-9241-43d0-9808-bde56f9b472e\") "
Jul 10 02:55:24.563572 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:24.563478    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79438ef9-9241-43d0-9808-bde56f9b472e-kube-api-access-xphm5" (OuterVolumeSpecName: "kube-api-access-xphm5") pod "79438ef9-9241-43d0-9808-bde56f9b472e" (UID: "79438ef9-9241-43d0-9808-bde56f9b472e"). InnerVolumeSpecName "kube-api-access-xphm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:55:24.565777 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:24.565754    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7523-140119920548720"]
Jul 10 02:55:24.569484 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:24.569468    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7523-140119920548720"]
Jul 10 02:55:24.662725 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:24.662642    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xphm5\" (UniqueName: \"kubernetes.io/projected/79438ef9-9241-43d0-9808-bde56f9b472e-kube-api-access-xphm5\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:25.377361 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:25.377334    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="118dc1153ace92820658aaefbb737a36a05f51d97c635de7bb55eb3aa5a47e26"
Jul 10 02:55:25.377361 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:25.377344    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7523-140119920548720"
Jul 10 02:55:25.386695 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:55:25.386670    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7523-140119920548720\" is forbidden: User \"system:node:ip-10-0-132-62.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-132-62.ec2.internal' and this object" podUID="79438ef9-9241-43d0-9808-bde56f9b472e" pod="openshift-ingress/test-curl-7523-140119920548720"
Jul 10 02:55:25.566167 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:25.566137    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79438ef9-9241-43d0-9808-bde56f9b472e" path="/var/lib/kubelet/pods/79438ef9-9241-43d0-9808-bde56f9b472e/volumes"
Jul 10 02:55:25.593046 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:25.593016    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"]
Jul 10 02:55:25.593245 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:25.593219    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx" podUID="1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0" containerName="maas-api" containerID="cri-o://ea69ad4694f2bc786e01848e2a3de5dff7272ec91cb457caae842e1de09e92d9" gracePeriod=30
Jul 10 02:55:25.842793 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:25.842768    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"
Jul 10 02:55:25.871213 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:25.871173    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-openshift-service-ca\") pod \"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0\" (UID: \"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0\") "
Jul 10 02:55:25.871378 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:25.871225    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tb25\" (UniqueName: \"kubernetes.io/projected/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-kube-api-access-4tb25\") pod \"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0\" (UID: \"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0\") "
Jul 10 02:55:25.871378 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:25.871284    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-maas-api-tls\") pod \"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0\" (UID: \"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0\") "
Jul 10 02:55:25.871589 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:25.871567    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0" (UID: "1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:55:25.873405 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:25.873382    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0" (UID: "1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:55:25.873542 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:25.873480    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-kube-api-access-4tb25" (OuterVolumeSpecName: "kube-api-access-4tb25") pod "1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0" (UID: "1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0"). InnerVolumeSpecName "kube-api-access-4tb25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:55:25.972357 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:25.972275    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-maas-api-tls\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:25.972357 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:25.972303    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-openshift-service-ca\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:25.972357 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:25.972315    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tb25\" (UniqueName: \"kubernetes.io/projected/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0-kube-api-access-4tb25\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:26.383147 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:26.383113    2579 generic.go:356] "Generic (PLEG): container finished" podID="1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0" containerID="ea69ad4694f2bc786e01848e2a3de5dff7272ec91cb457caae842e1de09e92d9" exitCode=0
Jul 10 02:55:26.383290 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:26.383169    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"
Jul 10 02:55:26.383290 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:26.383180    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx" event={"ID":"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0","Type":"ContainerDied","Data":"ea69ad4694f2bc786e01848e2a3de5dff7272ec91cb457caae842e1de09e92d9"}
Jul 10 02:55:26.383290 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:26.383246    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx" event={"ID":"1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0","Type":"ContainerDied","Data":"583ea3c25e0dbdeacb0e75b198158f2446183f019a75a94b7c8b7d0b672ca80a"}
Jul 10 02:55:26.383290 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:26.383267    2579 scope.go:117] "RemoveContainer" containerID="ea69ad4694f2bc786e01848e2a3de5dff7272ec91cb457caae842e1de09e92d9"
Jul 10 02:55:26.391941 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:26.391922    2579 scope.go:117] "RemoveContainer" containerID="ea69ad4694f2bc786e01848e2a3de5dff7272ec91cb457caae842e1de09e92d9"
Jul 10 02:55:26.392231 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:55:26.392210    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea69ad4694f2bc786e01848e2a3de5dff7272ec91cb457caae842e1de09e92d9\": container with ID starting with ea69ad4694f2bc786e01848e2a3de5dff7272ec91cb457caae842e1de09e92d9 not found: ID does not exist" containerID="ea69ad4694f2bc786e01848e2a3de5dff7272ec91cb457caae842e1de09e92d9"
Jul 10 02:55:26.392274 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:26.392239    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea69ad4694f2bc786e01848e2a3de5dff7272ec91cb457caae842e1de09e92d9"} err="failed to get container status \"ea69ad4694f2bc786e01848e2a3de5dff7272ec91cb457caae842e1de09e92d9\": rpc error: code = NotFound desc = could not find container \"ea69ad4694f2bc786e01848e2a3de5dff7272ec91cb457caae842e1de09e92d9\": container with ID starting with ea69ad4694f2bc786e01848e2a3de5dff7272ec91cb457caae842e1de09e92d9 not found: ID does not exist"
Jul 10 02:55:26.403787 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:26.403757    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"]
Jul 10 02:55:26.407468 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:26.407447    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e239ac-677458755b-wdbpx"]
Jul 10 02:55:27.566472 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:27.566437    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0" path="/var/lib/kubelet/pods/1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0/volumes"
Jul 10 02:55:29.058655 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:29.058622    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:55:32.905862 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:32.905830    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"]
Jul 10 02:55:32.906249 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:32.906075    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz" podUID="d0601606-d77f-4c2f-9f90-b99fe089a08f" containerName="maas-api" containerID="cri-o://c7dd4373e0ec095f63ac6f6506930274cdcd1054aafc0c5550f841146be49589" gracePeriod=30
Jul 10 02:55:33.348157 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.348132    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:55:33.411240 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.411206    2579 generic.go:356] "Generic (PLEG): container finished" podID="d0601606-d77f-4c2f-9f90-b99fe089a08f" containerID="c7dd4373e0ec095f63ac6f6506930274cdcd1054aafc0c5550f841146be49589" exitCode=0
Jul 10 02:55:33.411402 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.411248    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"
Jul 10 02:55:33.411402 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.411273    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz" event={"ID":"d0601606-d77f-4c2f-9f90-b99fe089a08f","Type":"ContainerDied","Data":"c7dd4373e0ec095f63ac6f6506930274cdcd1054aafc0c5550f841146be49589"}
Jul 10 02:55:33.411402 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.411315    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz" event={"ID":"d0601606-d77f-4c2f-9f90-b99fe089a08f","Type":"ContainerDied","Data":"46f77f6c76d2cc6712d130d0001553019ae58db0ebbf1ca8d193a8d183ba35c2"}
Jul 10 02:55:33.411402 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.411337    2579 scope.go:117] "RemoveContainer" containerID="c7dd4373e0ec095f63ac6f6506930274cdcd1054aafc0c5550f841146be49589"
Jul 10 02:55:33.421087 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.421062    2579 scope.go:117] "RemoveContainer" containerID="c7dd4373e0ec095f63ac6f6506930274cdcd1054aafc0c5550f841146be49589"
Jul 10 02:55:33.421364 ip-10-0-132-62 kubenswrapper[2579]: E0710 02:55:33.421336    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7dd4373e0ec095f63ac6f6506930274cdcd1054aafc0c5550f841146be49589\": container with ID starting with c7dd4373e0ec095f63ac6f6506930274cdcd1054aafc0c5550f841146be49589 not found: ID does not exist" containerID="c7dd4373e0ec095f63ac6f6506930274cdcd1054aafc0c5550f841146be49589"
Jul 10 02:55:33.421426 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.421377    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7dd4373e0ec095f63ac6f6506930274cdcd1054aafc0c5550f841146be49589"} err="failed to get container status \"c7dd4373e0ec095f63ac6f6506930274cdcd1054aafc0c5550f841146be49589\": rpc error: code = NotFound desc = could not find container \"c7dd4373e0ec095f63ac6f6506930274cdcd1054aafc0c5550f841146be49589\": container with ID starting with c7dd4373e0ec095f63ac6f6506930274cdcd1054aafc0c5550f841146be49589 not found: ID does not exist"
Jul 10 02:55:33.433151 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.433091    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0601606-d77f-4c2f-9f90-b99fe089a08f-maas-api-tls\") pod \"d0601606-d77f-4c2f-9f90-b99fe089a08f\" (UID: \"d0601606-d77f-4c2f-9f90-b99fe089a08f\") "
Jul 10 02:55:33.433151 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.433125    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sqlw\" (UniqueName: \"kubernetes.io/projected/d0601606-d77f-4c2f-9f90-b99fe089a08f-kube-api-access-8sqlw\") pod \"d0601606-d77f-4c2f-9f90-b99fe089a08f\" (UID: \"d0601606-d77f-4c2f-9f90-b99fe089a08f\") "
Jul 10 02:55:33.433295 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.433218    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0601606-d77f-4c2f-9f90-b99fe089a08f-openshift-service-ca\") pod \"d0601606-d77f-4c2f-9f90-b99fe089a08f\" (UID: \"d0601606-d77f-4c2f-9f90-b99fe089a08f\") "
Jul 10 02:55:33.433606 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.433583    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0601606-d77f-4c2f-9f90-b99fe089a08f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d0601606-d77f-4c2f-9f90-b99fe089a08f" (UID: "d0601606-d77f-4c2f-9f90-b99fe089a08f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:55:33.435204 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.435182    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0601606-d77f-4c2f-9f90-b99fe089a08f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d0601606-d77f-4c2f-9f90-b99fe089a08f" (UID: "d0601606-d77f-4c2f-9f90-b99fe089a08f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:55:33.435278 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.435260    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0601606-d77f-4c2f-9f90-b99fe089a08f-kube-api-access-8sqlw" (OuterVolumeSpecName: "kube-api-access-8sqlw") pod "d0601606-d77f-4c2f-9f90-b99fe089a08f" (UID: "d0601606-d77f-4c2f-9f90-b99fe089a08f"). InnerVolumeSpecName "kube-api-access-8sqlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:55:33.533940 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.533890    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0601606-d77f-4c2f-9f90-b99fe089a08f-openshift-service-ca\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:33.533940 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.533931    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0601606-d77f-4c2f-9f90-b99fe089a08f-maas-api-tls\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:33.533940 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.533941    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sqlw\" (UniqueName: \"kubernetes.io/projected/d0601606-d77f-4c2f-9f90-b99fe089a08f-kube-api-access-8sqlw\") on node \"ip-10-0-132-62.ec2.internal\" DevicePath \"\""
Jul 10 02:55:33.727043 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.726968    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"]
Jul 10 02:55:33.732299 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:33.732274    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-e812a4-5459967b49-sx7tz"]
Jul 10 02:55:35.565258 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:35.565226    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0601606-d77f-4c2f-9f90-b99fe089a08f" path="/var/lib/kubelet/pods/d0601606-d77f-4c2f-9f90-b99fe089a08f/volumes"
Jul 10 02:55:39.626181 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:39.626147    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-77fd4fbdbc-2z4nv_38d7910d-0e56-47b8-ac09-155be82b3034/authorino/0.log"
Jul 10 02:55:42.161116 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:42.161083    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8j9dt"]
Jul 10 02:55:45.743807 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:45.743782    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94f6fbcc5-jszsk_b75e848a-4e89-46f3-bdb4-7224a110f1c1/copy-manifests/0.log"
Jul 10 02:55:45.750902 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:45.750867    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94f6fbcc5-jszsk_b75e848a-4e89-46f3-bdb4-7224a110f1c1/manager/0.log"
Jul 10 02:55:47.569460 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:47.569427    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-77fd4fbdbc-2z4nv_38d7910d-0e56-47b8-ac09-155be82b3034/authorino/0.log"
Jul 10 02:55:48.309961 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:48.309929    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8j9dt_68a64a2d-d11d-4fa1-9bf7-2473052c3d32/limitador/0.log"
Jul 10 02:55:49.168703 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:49.168675    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779d665c94-qbtkd_9602c94d-88e4-47f4-af44-562d46baa3b8/kube-auth-proxy/0.log"
Jul 10 02:55:50.633828 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:50.633804    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-k66z7_e5000069-9d9e-4358-b68c-c7489c292b12/main/0.log"
Jul 10 02:55:54.939970 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:54.939938    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hdm6f/must-gather-kcbsx"]
Jul 10 02:55:54.940342 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:54.940195    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0601606-d77f-4c2f-9f90-b99fe089a08f" containerName="maas-api"
Jul 10 02:55:54.940342 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:54.940206    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0601606-d77f-4c2f-9f90-b99fe089a08f" containerName="maas-api"
Jul 10 02:55:54.940342 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:54.940222    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0" containerName="maas-api"
Jul 10 02:55:54.940342 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:54.940228    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0" containerName="maas-api"
Jul 10 02:55:54.940342 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:54.940238    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79438ef9-9241-43d0-9808-bde56f9b472e" containerName="test-curl-7523-140119920548720"
Jul 10 02:55:54.940342 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:54.940245    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="79438ef9-9241-43d0-9808-bde56f9b472e" containerName="test-curl-7523-140119920548720"
Jul 10 02:55:54.940342 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:54.940284    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ea8e0a5-dd16-4ba1-82e7-9cb41a53b1d0" containerName="maas-api"
Jul 10 02:55:54.940342 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:54.940291    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0601606-d77f-4c2f-9f90-b99fe089a08f" containerName="maas-api"
Jul 10 02:55:54.940342 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:54.940298    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="79438ef9-9241-43d0-9808-bde56f9b472e" containerName="test-curl-7523-140119920548720"
Jul 10 02:55:54.943263 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:54.943243    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hdm6f/must-gather-kcbsx"
Jul 10 02:55:54.945987 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:54.945966    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hdm6f\"/\"default-dockercfg-zhgkd\""
Jul 10 02:55:54.946171 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:54.946152    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hdm6f\"/\"openshift-service-ca.crt\""
Jul 10 02:55:54.947202 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:54.947187    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hdm6f\"/\"kube-root-ca.crt\""
Jul 10 02:55:54.950368 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:54.950263    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hdm6f/must-gather-kcbsx"]
Jul 10 02:55:55.002713 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:55.002690    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj98d\" (UniqueName: \"kubernetes.io/projected/c95c9b6e-7f1e-444e-98b1-7984f3ed6a4c-kube-api-access-tj98d\") pod \"must-gather-kcbsx\" (UID: \"c95c9b6e-7f1e-444e-98b1-7984f3ed6a4c\") " pod="openshift-must-gather-hdm6f/must-gather-kcbsx"
Jul 10 02:55:55.002816 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:55.002722    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c95c9b6e-7f1e-444e-98b1-7984f3ed6a4c-must-gather-output\") pod \"must-gather-kcbsx\" (UID: \"c95c9b6e-7f1e-444e-98b1-7984f3ed6a4c\") " pod="openshift-must-gather-hdm6f/must-gather-kcbsx"
Jul 10 02:55:55.103740 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:55.103711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj98d\" (UniqueName: \"kubernetes.io/projected/c95c9b6e-7f1e-444e-98b1-7984f3ed6a4c-kube-api-access-tj98d\") pod \"must-gather-kcbsx\" (UID: \"c95c9b6e-7f1e-444e-98b1-7984f3ed6a4c\") " pod="openshift-must-gather-hdm6f/must-gather-kcbsx"
Jul 10 02:55:55.103859 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:55.103754    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c95c9b6e-7f1e-444e-98b1-7984f3ed6a4c-must-gather-output\") pod \"must-gather-kcbsx\" (UID: \"c95c9b6e-7f1e-444e-98b1-7984f3ed6a4c\") " pod="openshift-must-gather-hdm6f/must-gather-kcbsx"
Jul 10 02:55:55.104131 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:55.104115    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c95c9b6e-7f1e-444e-98b1-7984f3ed6a4c-must-gather-output\") pod \"must-gather-kcbsx\" (UID: \"c95c9b6e-7f1e-444e-98b1-7984f3ed6a4c\") " pod="openshift-must-gather-hdm6f/must-gather-kcbsx"
Jul 10 02:55:55.118971 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:55.118949    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj98d\" (UniqueName: \"kubernetes.io/projected/c95c9b6e-7f1e-444e-98b1-7984f3ed6a4c-kube-api-access-tj98d\") pod \"must-gather-kcbsx\" (UID: \"c95c9b6e-7f1e-444e-98b1-7984f3ed6a4c\") " pod="openshift-must-gather-hdm6f/must-gather-kcbsx"
Jul 10 02:55:55.252828 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:55.252735    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hdm6f/must-gather-kcbsx"
Jul 10 02:55:55.379467 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:55.379346    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hdm6f/must-gather-kcbsx"]
Jul 10 02:55:55.382091 ip-10-0-132-62 kubenswrapper[2579]: W0710 02:55:55.382059    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc95c9b6e_7f1e_444e_98b1_7984f3ed6a4c.slice/crio-8a9bc63d00cc96426109fa078d902563d4b60fe1926d079ab7c382ed79d0e2ac WatchSource:0}: Error finding container 8a9bc63d00cc96426109fa078d902563d4b60fe1926d079ab7c382ed79d0e2ac: Status 404 returned error can't find the container with id 8a9bc63d00cc96426109fa078d902563d4b60fe1926d079ab7c382ed79d0e2ac
Jul 10 02:55:55.497766 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:55.497733    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hdm6f/must-gather-kcbsx" event={"ID":"c95c9b6e-7f1e-444e-98b1-7984f3ed6a4c","Type":"ContainerStarted","Data":"8a9bc63d00cc96426109fa078d902563d4b60fe1926d079ab7c382ed79d0e2ac"}
Jul 10 02:55:56.504371 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:56.504331    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hdm6f/must-gather-kcbsx" event={"ID":"c95c9b6e-7f1e-444e-98b1-7984f3ed6a4c","Type":"ContainerStarted","Data":"eca5fece88c8108419d1c6f6db94ab6b7e429c0817d44bef4b172abdd040e7ed"}
Jul 10 02:55:56.504842 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:56.504383    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hdm6f/must-gather-kcbsx" event={"ID":"c95c9b6e-7f1e-444e-98b1-7984f3ed6a4c","Type":"ContainerStarted","Data":"3a428c9b7ba94e645d572730e532b350eda19db40c4de99e2ae70e2bbaadc677"}
Jul 10 02:55:56.524257 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:56.524205    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hdm6f/must-gather-kcbsx" podStartSLOduration=1.860663688 podStartE2EDuration="2.524190424s" totalImagesPullingTime="663.526736ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:55:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:55:56.522388144 +0000 UTC m=+2503.505145831" watchObservedRunningTime="2026-07-10 02:55:56.524190424 +0000 UTC m=+2503.506948088"
Jul 10 02:55:57.633106 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:57.633077    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w59q2_95280b80-3f30-46c4-be9b-33ff86d5ef24/global-pull-secret-syncer/0.log"
Jul 10 02:55:57.750375 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:57.750347    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wgkcm_67f488cd-d334-4a42-b235-29879ebe70e0/konnectivity-agent/0.log"
Jul 10 02:55:57.770453 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:55:57.770426    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-62.ec2.internal_3593c0baaa5f598fd63dfa27adb409e3/haproxy/0.log"
Jul 10 02:56:02.823581 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:02.823187    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-77fd4fbdbc-2z4nv_38d7910d-0e56-47b8-ac09-155be82b3034/authorino/0.log"
Jul 10 02:56:03.117257 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:03.117230    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8j9dt_68a64a2d-d11d-4fa1-9bf7-2473052c3d32/limitador/0.log"
Jul 10 02:56:05.299590 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:05.299476    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rbcdz_700a1c43-92a0-4541-84fc-5dce3b0a5307/node-exporter/0.log"
Jul 10 02:56:05.322442 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:05.322410    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rbcdz_700a1c43-92a0-4541-84fc-5dce3b0a5307/kube-rbac-proxy/0.log"
Jul 10 02:56:05.360409 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:05.360308    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rbcdz_700a1c43-92a0-4541-84fc-5dce3b0a5307/init-textfile/0.log"
Jul 10 02:56:07.182202 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.182160    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-7fqxc_a07b77b4-3144-4cbb-8937-0a6bd23da2c0/networking-console-plugin/0.log"
Jul 10 02:56:07.594860 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.594778    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"]
Jul 10 02:56:07.600198 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.600173    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.609486 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.609458    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"]
Jul 10 02:56:07.742868 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.742813    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f41755f-6c2a-4ca3-82a8-42e4a4c138ad-podres\") pod \"perf-node-gather-daemonset-tdpcg\" (UID: \"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.743090 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.743073    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f41755f-6c2a-4ca3-82a8-42e4a4c138ad-proc\") pod \"perf-node-gather-daemonset-tdpcg\" (UID: \"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.743188 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.743176    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-475vb\" (UniqueName: \"kubernetes.io/projected/6f41755f-6c2a-4ca3-82a8-42e4a4c138ad-kube-api-access-475vb\") pod \"perf-node-gather-daemonset-tdpcg\" (UID: \"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.743367 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.743343    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f41755f-6c2a-4ca3-82a8-42e4a4c138ad-lib-modules\") pod \"perf-node-gather-daemonset-tdpcg\" (UID: \"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.743418 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.743388    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f41755f-6c2a-4ca3-82a8-42e4a4c138ad-sys\") pod \"perf-node-gather-daemonset-tdpcg\" (UID: \"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.844942 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.844855    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f41755f-6c2a-4ca3-82a8-42e4a4c138ad-proc\") pod \"perf-node-gather-daemonset-tdpcg\" (UID: \"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.844942 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.844938    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-475vb\" (UniqueName: \"kubernetes.io/projected/6f41755f-6c2a-4ca3-82a8-42e4a4c138ad-kube-api-access-475vb\") pod \"perf-node-gather-daemonset-tdpcg\" (UID: \"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.845187 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.844989    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6f41755f-6c2a-4ca3-82a8-42e4a4c138ad-proc\") pod \"perf-node-gather-daemonset-tdpcg\" (UID: \"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.845187 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.845033    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f41755f-6c2a-4ca3-82a8-42e4a4c138ad-lib-modules\") pod \"perf-node-gather-daemonset-tdpcg\" (UID: \"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.845187 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.845082    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f41755f-6c2a-4ca3-82a8-42e4a4c138ad-sys\") pod \"perf-node-gather-daemonset-tdpcg\" (UID: \"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.845187 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.845150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f41755f-6c2a-4ca3-82a8-42e4a4c138ad-podres\") pod \"perf-node-gather-daemonset-tdpcg\" (UID: \"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.845407 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.845343    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6f41755f-6c2a-4ca3-82a8-42e4a4c138ad-podres\") pod \"perf-node-gather-daemonset-tdpcg\" (UID: \"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.845468 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.845426    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f41755f-6c2a-4ca3-82a8-42e4a4c138ad-sys\") pod \"perf-node-gather-daemonset-tdpcg\" (UID: \"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.845468 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.845455    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f41755f-6c2a-4ca3-82a8-42e4a4c138ad-lib-modules\") pod \"perf-node-gather-daemonset-tdpcg\" (UID: \"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.853537 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.853488    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-475vb\" (UniqueName: \"kubernetes.io/projected/6f41755f-6c2a-4ca3-82a8-42e4a4c138ad-kube-api-access-475vb\") pod \"perf-node-gather-daemonset-tdpcg\" (UID: \"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad\") " pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:07.917495 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:07.917467    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:08.085009 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:08.084975    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"]
Jul 10 02:56:08.586943 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:08.586907    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg" event={"ID":"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad","Type":"ContainerStarted","Data":"82c0b8db39617e812dad458c9c5a4832b89b2ac5239a74bfce03815e29ae23b6"}
Jul 10 02:56:08.587363 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:08.586968    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg" event={"ID":"6f41755f-6c2a-4ca3-82a8-42e4a4c138ad","Type":"ContainerStarted","Data":"57a4be5a74dd4d104ca0472b111d483d6165d80901afa9367211ce2f178d682b"}
Jul 10 02:56:08.587363 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:08.586994    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:08.604306 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:08.604243    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg" podStartSLOduration=1.604221736 podStartE2EDuration="1.604221736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:56:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:56:08.602313292 +0000 UTC m=+2515.585070976" watchObservedRunningTime="2026-07-10 02:56:08.604221736 +0000 UTC m=+2515.586979424"
Jul 10 02:56:09.883524 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:09.883477    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f7sbz_5b633a59-0e3c-45eb-990d-5a55827eefe1/dns/0.log"
Jul 10 02:56:09.904022 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:09.903995    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f7sbz_5b633a59-0e3c-45eb-990d-5a55827eefe1/kube-rbac-proxy/0.log"
Jul 10 02:56:10.027109 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:10.027072    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9gnrj_cdca22f8-36eb-4162-abd9-9fed3a472a58/dns-node-resolver/0.log"
Jul 10 02:56:10.600769 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:10.600736    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6f7d7fccf8-ps2l5_90897a69-5317-40c8-9536-1221271d7623/registry/0.log"
Jul 10 02:56:10.669088 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:10.669057    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mnqnz_79684caa-c963-4b62-8a74-ed4e87d3f201/node-ca/0.log"
Jul 10 02:56:11.747705 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:11.747676    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779d665c94-qbtkd_9602c94d-88e4-47f4-af44-562d46baa3b8/kube-auth-proxy/0.log"
Jul 10 02:56:12.577055 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:12.577016    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qj2g2_6b753a26-89e1-4b05-8a70-d922cf975723/serve-healthcheck-canary/0.log"
Jul 10 02:56:13.198497 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:13.198468    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vg6lr_de37ac86-4744-475f-bfac-66c48f109148/kube-rbac-proxy/0.log"
Jul 10 02:56:13.220290 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:13.220264    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vg6lr_de37ac86-4744-475f-bfac-66c48f109148/exporter/0.log"
Jul 10 02:56:13.240878 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:13.240843    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vg6lr_de37ac86-4744-475f-bfac-66c48f109148/extractor/0.log"
Jul 10 02:56:14.603035 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:14.603002    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hdm6f/perf-node-gather-daemonset-tdpcg"
Jul 10 02:56:15.826522 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:15.826471    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94f6fbcc5-jszsk_b75e848a-4e89-46f3-bdb4-7224a110f1c1/manager/0.log"
Jul 10 02:56:15.847853 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:15.847830    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94f6fbcc5-jszsk_b75e848a-4e89-46f3-bdb4-7224a110f1c1/copy-manifests/0.log"
Jul 10 02:56:17.190934 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:17.190897    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-865944fd75-qj999_7e66e036-1b0f-4b5d-9555-4b5f356ef186/manager/0.log"
Jul 10 02:56:18.393020 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:18.392978    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779d665c94-qbtkd_9602c94d-88e4-47f4-af44-562d46baa3b8/kube-auth-proxy/0.log"
Jul 10 02:56:19.895053 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:19.895015    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779d665c94-qbtkd_9602c94d-88e4-47f4-af44-562d46baa3b8/kube-auth-proxy/0.log"
Jul 10 02:56:22.282773 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:22.282741    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-tfktk_95b3336b-3d04-4991-9c18-a95e7cc1661f/migrator/0.log"
Jul 10 02:56:22.305613 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:22.305588    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-tfktk_95b3336b-3d04-4991-9c18-a95e7cc1661f/graceful-termination/0.log"
Jul 10 02:56:23.612040 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:23.611958    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8dgnt_1007a2ac-fc5c-41b3-8b51-d1006101a828/kube-multus/0.log"
Jul 10 02:56:23.951713 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:23.951646    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m42l5_d64fac57-3910-4a19-8308-f56df2a18b8f/kube-multus-additional-cni-plugins/0.log"
Jul 10 02:56:23.971458 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:23.971437    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m42l5_d64fac57-3910-4a19-8308-f56df2a18b8f/egress-router-binary-copy/0.log"
Jul 10 02:56:23.990997 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:23.990969    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m42l5_d64fac57-3910-4a19-8308-f56df2a18b8f/cni-plugins/0.log"
Jul 10 02:56:24.013465 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:24.013441    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m42l5_d64fac57-3910-4a19-8308-f56df2a18b8f/bond-cni-plugin/0.log"
Jul 10 02:56:24.032881 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:24.032860    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m42l5_d64fac57-3910-4a19-8308-f56df2a18b8f/routeoverride-cni/0.log"
Jul 10 02:56:24.052783 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:24.052755    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m42l5_d64fac57-3910-4a19-8308-f56df2a18b8f/whereabouts-cni-bincopy/0.log"
Jul 10 02:56:24.073620 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:24.073596    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-m42l5_d64fac57-3910-4a19-8308-f56df2a18b8f/whereabouts-cni/0.log"
Jul 10 02:56:24.245726 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:24.245646    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dcc8z_fafe8b6d-e09b-4dac-9687-43f5c7265be7/network-metrics-daemon/0.log"
Jul 10 02:56:24.265912 ip-10-0-132-62 kubenswrapper[2579]: I0710 02:56:24.265892    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dcc8z_fafe8b6d-e09b-4dac-9687-43f5c7265be7/kube-rbac-proxy/0.log"