Jul 02 23:10:54.620414 ip-10-0-140-52 systemd[1]: Starting Kubernetes Kubelet...
Jul 02 23:10:55.067036 ip-10-0-140-52 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 23:10:55.067036 ip-10-0-140-52 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 02 23:10:55.067036 ip-10-0-140-52 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 23:10:55.067036 ip-10-0-140-52 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 02 23:10:55.067036 ip-10-0-140-52 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 02 23:10:55.067036 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.052209    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 02 23:10:55.067036 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056679    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 23:10:55.067036 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056692    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 23:10:55.067036 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056696    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 23:10:55.067036 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056702    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 23:10:55.067036 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056706    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 23:10:55.067036 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056710    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 23:10:55.067036 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056713    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056716    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056719    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056722    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056732    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056735    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056739    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056742    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056745    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056748    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056752    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056755    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056758    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056761    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056764    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056767    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056769    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056772    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056774    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056777    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056780    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 23:10:55.071231 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056783    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056786    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056788    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056790    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056793    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056795    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056799    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056801    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056804    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056807    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056809    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056812    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056814    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056817    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056819    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056822    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056825    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056828    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056830    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056833    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 23:10:55.077432 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056835    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056838    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056840    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056843    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056846    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056850    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056854    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056856    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056859    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056861    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056865    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056867    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056870    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056872    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056875    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056878    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056881    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056883    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056886    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056888    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 23:10:55.079829 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056891    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056893    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056895    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056898    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056900    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056902    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056905    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056908    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056911    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056914    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056916    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056919    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056921    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056925    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056929    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056932    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056936    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056939    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056942    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 23:10:55.083976 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056945    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056948    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056952    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056954    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056957    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056960    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056963    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056965    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056968    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056975    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056978    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056981    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.056984    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059054    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059062    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059067    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059070    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059073    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059076    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059079    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 23:10:55.088441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059082    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059085    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059089    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059092    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059095    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059097    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059100    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059103    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059106    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059109    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059112    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059114    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059134    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059138    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059141    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059144    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059147    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059150    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059152    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059155    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 23:10:55.092854 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059158    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059161    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059164    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059167    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059170    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059173    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059176    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059179    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059181    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059184    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059186    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059189    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059192    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059195    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059200    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059203    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059205    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059208    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059211    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059213    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 23:10:55.096916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059216    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059219    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059221    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059225    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059229    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059232    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059235    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059238    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059241    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059243    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059246    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059248    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059251    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059254    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059256    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059259    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059262    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059264    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059267    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 23:10:55.101207 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059269    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059272    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059274    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059278    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059281    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059284    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059287    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059289    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059292    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059294    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059297    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059299    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059302    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059304    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059307    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059309    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059312    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059315    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059317    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 23:10:55.105369 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059320    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059322    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059324    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059327    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059329    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059332    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059334    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059337    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059339    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059341    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059346    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059350    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059353    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059356    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059741    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059747    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059750    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059753    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059756    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 23:10:55.109718 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059760    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059763    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059766    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059769    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059772    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059774    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059778    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059781    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059784    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059787    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059790    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059792    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059795    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059798    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059801    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059803    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059806    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059808    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059811    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 23:10:55.116453 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059814    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059817    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059819    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059822    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059826    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059828    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059831    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059833    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059836    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059839    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059841    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059844    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059846    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059849    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059851    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059854    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059856    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059859    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059861    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 23:10:55.120486 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059864    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059866    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059869    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059872    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059874    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059877    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059880    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059883    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059885    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059888    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059891    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059895    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059898    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059900    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059903    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059906    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059908    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059911    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059914    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059916    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 23:10:55.124006 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059919    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059921    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059924    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059926    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059930    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059933    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059935    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059938    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059940    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059943    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059945    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059948    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059950    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059953    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059955    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059958    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059961    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059963    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059967    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059969    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 23:10:55.128697 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059971    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059974    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059977    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059981    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059983    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059986    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059988    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059991    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059993    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059995    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.059999    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060008    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060011    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060013    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060016    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060018    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060091    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060098    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060105    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060111    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060132    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 02 23:10:55.132771 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060137    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060144    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060150    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060154    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060157    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060160    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060163    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060166    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060170    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060173    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060175    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060178    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060181    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060185    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060190    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060193    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060196    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060200    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060204    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060207    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060210    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060213    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060216    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060220    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 02 23:10:55.137466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060224    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060227    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060230    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060236    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060239    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060242    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060244    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060247    2580 flags.go:64] FLAG: --enable-server="true"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060250    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060255    2580 flags.go:64] FLAG: --event-burst="100"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060258    2580 flags.go:64] FLAG: --event-qps="50"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060261    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060264    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060267    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060271    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060273    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060277    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060280    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060283    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060286    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060289    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060291    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060295    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060298    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060301    2580 flags.go:64] FLAG: --feature-gates=""
Jul 02 23:10:55.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060305    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 02 23:10:55.140426 ip-10-0-140-52 systemd[1]: Started Kubernetes Kubelet.
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060308    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060311    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060314    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060317    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060320    2580 flags.go:64] FLAG: --help="false"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060323    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-140-52.ec2.internal"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060327    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060329    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060332    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060337    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060340    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060343    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060346    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060349    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060351    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060354    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060357    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060360    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060363    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060366    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060369    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060372    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060374    2580 flags.go:64] FLAG: --lock-file=""
Jul 02 23:10:55.153048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060377    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060380    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060383    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060388    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060391    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060395    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060398    2580 flags.go:64] FLAG: --logging-format="text"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060401    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060405    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060407    2580 flags.go:64] FLAG: --manifest-url=""
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060410    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060415    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060418    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060422    2580 flags.go:64] FLAG: --max-pods="110"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060425    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060427    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060431    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060433    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060438    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060441    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060444    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060451    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060453    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060456    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 02 23:10:55.184371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060460    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060462    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060468    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060470    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060473    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060476    2580 flags.go:64] FLAG: --port="10250"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060479    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060482    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0380b7c9afa3362fd"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060485    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060488    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060491    2580 flags.go:64] FLAG: --register-node="true"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060494    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060498    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060502    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060505    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060508    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060512    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060515    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060518    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060521    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060524    2580 flags.go:64] FLAG: --runonce="false"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060527    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060530    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060533    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060536    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 02 23:10:55.190375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060540    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060543    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060547    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060550    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060553    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060556    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060558    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060561    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060564    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060567    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060570    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060576    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060578    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060581    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060585    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060588    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060591    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060594    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060597    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060600    2580 flags.go:64] FLAG: --v="2"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060606    2580 flags.go:64] FLAG: --version="false"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060609    2580 flags.go:64] FLAG: --vmodule=""
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060613    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.060616    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 02 23:10:55.350641 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060740    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060745    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060749    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060752    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060756    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060758    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060761    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060764    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060767    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060774    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060778    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060781    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060785    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060789    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060791    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060794    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060797    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060799    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060802    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060804    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 23:10:55.369074 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060807    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060810    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060812    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060815    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060817    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060820    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060822    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060825    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060829    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060832    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060834    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060837    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060839    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060842    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060844    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060848    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060851    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060854    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060857    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 23:10:55.370131 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060860    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060863    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060865    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060870    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060873    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060876    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060879    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060881    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060884    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060886    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060889    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060891    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060894    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060896    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060899    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060902    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060904    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060906    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060909    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060911    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 23:10:55.372021 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060914    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060918    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060920    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060923    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060925    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060928    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060930    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060933    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060935    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060938    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060940    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060943    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060945    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060948    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060950    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060954    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060957    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060960    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060962    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060965    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 23:10:55.378709 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060968    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060970    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060973    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060975    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060978    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060980    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060983    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060985    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060988    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060990    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060993    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060995    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.060998    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.061002    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.061005    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.061007    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.061010    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.061012    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.061015    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 23:10:55.381213 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.061017    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 23:10:55.385373 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.061851    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 23:10:55.385373 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.071147    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 02 23:10:55.385373 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.071164    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 02 23:10:55.385373 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.071923    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 02 23:10:55.385373 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.075145    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 02 23:10:55.385373 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075220    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 23:10:55.385373 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075226    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 23:10:55.385373 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075230    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 23:10:55.385373 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075234    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 23:10:55.385373 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075237    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 23:10:55.385373 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075240    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 23:10:55.385373 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075242    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 23:10:55.385373 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075245    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 23:10:55.385373 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075248    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 23:10:55.385373 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075251    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075253    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075256    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075259    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075261    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075264    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075269    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075273    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075276    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075279    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075282    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075285    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075288    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075291    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075293    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075296    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075299    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075302    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075304    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 23:10:55.386595 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075307    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075313    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075316    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075319    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075321    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075324    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075327    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075329    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075332    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075334    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075337    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075340    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075343    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075345    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075348    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075351    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075354    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075357    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075360    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075363    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 23:10:55.399225 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075365    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075368    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075371    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075373    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075376    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075378    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075381    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075383    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075386    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075389    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075391    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075394    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075397    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075400    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075402    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075406    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075408    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075411    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075413    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075416    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 23:10:55.400012 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075418    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075421    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075424    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075426    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075429    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075431    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075434    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075437    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075439    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075442    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075444    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075447    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075450    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075452    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075455    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075457    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075460    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075463    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075466    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075468    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 23:10:55.402033 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075473    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 23:10:55.402865 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075476    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 23:10:55.402865 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075479    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 23:10:55.402865 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075482    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 23:10:55.402865 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075485    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 23:10:55.402865 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075489    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 23:10:55.402865 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075494    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 23:10:55.402865 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075497    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 23:10:55.402865 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075501    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 23:10:55.402865 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075504    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 23:10:55.402865 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075507    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 23:10:55.402865 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.075512    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 23:10:55.402865 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075630    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 02 23:10:55.402865 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075635    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 02 23:10:55.402865 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075639    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075642    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075645    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075648    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075651    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075656    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075659    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075662    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075664    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075667    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075671    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075674    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075677    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075679    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075682    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075685    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075687    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075690    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075693    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 02 23:10:55.404489 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075697    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075699    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075702    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075704    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075707    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075710    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075712    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075715    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075718    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075721    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075723    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075726    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075728    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075731    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075734    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075736    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075739    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075742    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075744    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 02 23:10:55.405299 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075747    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075750    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075753    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075755    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075757    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075761    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075763    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075767    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075771    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075774    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075777    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075781    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075784    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075788    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075791    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075794    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075796    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075799    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075802    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075804    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 02 23:10:55.407746 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075807    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075810    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075812    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075815    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075818    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075820    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075823    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075825    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075828    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075831    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075833    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075835    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075838    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075840    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075843    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075845    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075848    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075852    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075856    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075860    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 02 23:10:55.408559 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075863    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075866    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075869    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075871    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075875    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075878    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075881    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075884    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075887    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075890    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075893    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075896    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075898    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075901    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075903    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075906    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075909    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075911    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 02 23:10:55.411834 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:55.075913    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 02 23:10:55.412635 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.075919    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 02 23:10:55.412635 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.076051    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 02 23:10:55.412635 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.078952    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 02 23:10:55.412635 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.080343    2580 server.go:1019] "Starting client certificate rotation"
Jul 02 23:10:55.412635 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.082439    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 23:10:55.412635 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.083923    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 23:10:55.412635 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.083980    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 02 23:10:55.412635 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.085517    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 02 23:10:55.412635 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.096729    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 02 23:10:55.412635 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.101313    2580 log.go:25] "Validated CRI v1 image API"
Jul 02 23:10:55.412635 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.102341    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 02 23:10:55.412635 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.106591    2580 fs.go:135] Filesystem UUIDs: map[02ea3b5a-c383-46b6-823c-34ed13ed082d:/dev/nvme0n1p4 5d30a0b4-bdc3-4a06-a359-bb59911b94bd:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 02 23:10:55.415083 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.106618    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 02 23:10:55.415505 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.113207    2580 manager.go:216] Machine: {Timestamp:2026-07-02 23:10:55.111748248 +0000 UTC m=+0.365538239 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100618 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e307603524791f5c942208497b9ab SystemUUID:ec2e3076-0352-4791-f5c9-42208497b9ab BootID:07e544d3-aa25-4c1e-a341-7c7b1d27dd41 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:74:16:8f:5b:95 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:74:16:8f:5b:95 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:21:13:3e:e7:d5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 02 23:10:55.415505 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.113309    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 02 23:10:55.415505 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.113388    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 02 23:10:55.415505 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.116397    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 02 23:10:55.415505 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.116424    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-52.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 02 23:10:55.415505 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.117814    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 02 23:10:55.415505 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.117830    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 02 23:10:55.415505 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.117837    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.118443    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.119267    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.120599    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.124091    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.124533    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.128948    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.128966    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.128990    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.129007    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.129028    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.129474    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.129508    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.132087    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.134041    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.134068    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.135220    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.135235    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.135242    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.135248    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.135259    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.135267    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.135279    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.135284    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.135291    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.135297    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.135305    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.135310    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.136085    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.136092    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.138595    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-52.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.138608    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-52.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.138741    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.139606    2580 server.go:1268] "Started kubelet"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.139676    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.139683    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.139747    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.141854    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.141871    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 02 23:10:55.517721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.145000    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ndmwn"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.144436    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-52.ec2.internal.18be9bd826170a6c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-52.ec2.internal,UID:ip-10-0-140-52.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-52.ec2.internal,},FirstTimestamp:2026-07-02 23:10:55.139580524 +0000 UTC m=+0.393370513,LastTimestamp:2026-07-02 23:10:55.139580524 +0000 UTC m=+0.393370513,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-52.ec2.internal,}"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.146724    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.148728    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.148747    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.148804    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-52.ec2.internal\" not found"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.149503    2580 factory.go:55] Registering systemd factory
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.149518    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.149525    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.149525    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.149544    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.149625    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.149634    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.149878    2580 factory.go:153] Registering CRI-O factory
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.149890    2580 factory.go:223] Registration of the crio container factory successfully
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.149935    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.149963    2580 factory.go:103] Registering Raw factory
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.149976    2580 manager.go:1196] Started watching for new ooms in manager
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.153194    2580 manager.go:349] Starting recovery of all containers
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.157361    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ndmwn"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.190296    2580 manager.go:354] Recovery completed
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.190342    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.192552    2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-52.ec2.internal\" not found" node="ip-10-0-140-52.ec2.internal"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.191716    2580 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.198442    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.201089    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.201133    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.201149    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasSufficientPID"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.201691    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.201701    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.201720    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.201692    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.202749    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.202772    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 23:10:55.518941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.202785    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasSufficientPID"
Jul 02 23:10:55.523447 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.203209    2580 policy_none.go:49] "None policy: Start"
Jul 02 23:10:55.523447 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.203223    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 02 23:10:55.523447 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.203235    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 02 23:10:55.523447 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.204528    2580 policy_none.go:47] "Start"
Jul 02 23:10:55.523447 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.249241    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-52.ec2.internal\" not found"
Jul 02 23:10:55.523447 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.275363    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 02 23:10:55.523447 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.276588    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 02 23:10:55.523447 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.276625    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 02 23:10:55.523447 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.276650    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 02 23:10:55.523447 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.276689    2580 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Jul 02 23:10:55.523447 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.278884    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 23:10:55.523447 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.349308    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-52.ec2.internal\" not found"
Jul 02 23:10:55.523447 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.377728    2580 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Jul 02 23:10:55.523447 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.449916    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-52.ec2.internal\" not found"
Jul 02 23:10:55.646952 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.550239    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-52.ec2.internal\" not found"
Jul 02 23:10:55.646952 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.578440    2580 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Jul 02 23:10:55.651816 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.650918    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-52.ec2.internal\" not found"
Jul 02 23:10:55.759727 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.751260    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-52.ec2.internal\" not found"
Jul 02 23:10:55.861933 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.851549    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-52.ec2.internal\" not found"
Jul 02 23:10:55.918010 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.917998    2580 manager.go:337] "Starting Device Plugin manager"
Jul 02 23:10:55.918076 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.918030    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 02 23:10:55.918076 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.918043    2580 server.go:87] "Starting device plugin registration server"
Jul 02 23:10:55.918357 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.918345    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 02 23:10:55.918390 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.918360    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 02 23:10:55.918512 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.918433    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 02 23:10:55.918595 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.918510    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 02 23:10:55.918595 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.918521    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 02 23:10:55.919085 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.919067    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 02 23:10:55.919204 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:55.919135    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-52.ec2.internal\" not found"
Jul 02 23:10:55.979519 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.979470    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-52.ec2.internal"]
Jul 02 23:10:55.979670 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.979587    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 23:10:55.982232 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.982211    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 23:10:55.982322 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.982243    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 23:10:55.982322 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.982271    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasSufficientPID"
Jul 02 23:10:55.983535 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.983523    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 23:10:55.983659 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.983643    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal"
Jul 02 23:10:55.983689 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.983685    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 23:10:55.984550 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.984536    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 23:10:55.984611 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.984561    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 23:10:55.984611 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.984576    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasSufficientPID"
Jul 02 23:10:55.985192 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.985171    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 23:10:55.985251 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.985193    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 23:10:55.985251 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.985203    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasSufficientPID"
Jul 02 23:10:55.986283 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.986270    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-52.ec2.internal"
Jul 02 23:10:55.986321 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.986293    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 23:10:55.987137 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.987109    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 23:10:55.987196 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.987158    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 23:10:55.987196 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:55.987172    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasSufficientPID"
Jul 02 23:10:56.014826 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:56.014806    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-52.ec2.internal\" not found" node="ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.018965 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.018536    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 02 23:10:56.019777 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.019761    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasSufficientMemory"
Jul 02 23:10:56.019842 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.019789    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasNoDiskPressure"
Jul 02 23:10:56.019842 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.019799    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeHasSufficientPID"
Jul 02 23:10:56.019842 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.019817    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.020886 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:56.020873    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-52.ec2.internal\" not found" node="ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.026155 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.026142    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.026211 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:56.026160    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-52.ec2.internal\": node \"ip-10-0-140-52.ec2.internal\" not found"
Jul 02 23:10:56.030959 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.030942    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 23:10:56.036584 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:56.036567    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-52.ec2.internal\" not found"
Jul 02 23:10:56.053457 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.053432    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00f9bd621d84080c84f579c3e648158f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal\" (UID: \"00f9bd621d84080c84f579c3e648158f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.053552 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.053469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00f9bd621d84080c84f579c3e648158f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal\" (UID: \"00f9bd621d84080c84f579c3e648158f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.053552 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.053498    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5a709290dff23a904b036c05f5cc09e5-config\") pod \"kube-apiserver-proxy-ip-10-0-140-52.ec2.internal\" (UID: \"5a709290dff23a904b036c05f5cc09e5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.079539 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.079516    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 02 23:10:56.079885 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.079625    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 02 23:10:56.079885 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.079664    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 02 23:10:56.079885 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.079668    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 02 23:10:56.079885 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:56.079686    2580 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a49d69362784547be81a2b14b16aa655-13f01d68398eb4c1.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-140-52.ec2.internal.18be9bd829c1c7fc\": read tcp 10.0.140.52:35594->100.27.83.1:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-140-52.ec2.internal.18be9bd829c1c7fc  default   5926 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-52.ec2.internal,UID:ip-10-0-140-52.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-52.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-52.ec2.internal,},FirstTimestamp:2026-07-02 23:10:55 +0000 UTC,LastTimestamp:2026-07-02 23:10:56.019777513 +0000 UTC m=+1.273567501,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-52.ec2.internal,}"
Jul 02 23:10:56.136992 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:56.136960    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-52.ec2.internal\" not found"
Jul 02 23:10:56.153897 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.153873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5a709290dff23a904b036c05f5cc09e5-config\") pod \"kube-apiserver-proxy-ip-10-0-140-52.ec2.internal\" (UID: \"5a709290dff23a904b036c05f5cc09e5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.153962 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.153904    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00f9bd621d84080c84f579c3e648158f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal\" (UID: \"00f9bd621d84080c84f579c3e648158f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.153962 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.153925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00f9bd621d84080c84f579c3e648158f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal\" (UID: \"00f9bd621d84080c84f579c3e648158f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.154021 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.153967    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00f9bd621d84080c84f579c3e648158f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal\" (UID: \"00f9bd621d84080c84f579c3e648158f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.154021 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.153977    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00f9bd621d84080c84f579c3e648158f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal\" (UID: \"00f9bd621d84080c84f579c3e648158f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.154021 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.153977    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5a709290dff23a904b036c05f5cc09e5-config\") pod \"kube-apiserver-proxy-ip-10-0-140-52.ec2.internal\" (UID: \"5a709290dff23a904b036c05f5cc09e5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.159941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.159895    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 23:05:55 +0000 UTC" deadline="2027-12-13 04:52:39.406688299 +0000 UTC"
Jul 02 23:10:56.159941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.159941    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12677h41m43.246748942s"
Jul 02 23:10:56.169677 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.169634    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 23:10:56.249634 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.249606    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.260233 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.260197    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 23:10:56.261347 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.261333    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.268563 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.268541    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 02 23:10:56.319521 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.319484    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.324189 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.324172    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-52.ec2.internal"
Jul 02 23:10:56.815540 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:56.815503    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a709290dff23a904b036c05f5cc09e5.slice/crio-a3121edbbf8f4c968e9f14aa6738e05fb7b9bb62e3b74400be6fe9657255e7fd WatchSource:0}: Error finding container a3121edbbf8f4c968e9f14aa6738e05fb7b9bb62e3b74400be6fe9657255e7fd: Status 404 returned error can't find the container with id a3121edbbf8f4c968e9f14aa6738e05fb7b9bb62e3b74400be6fe9657255e7fd
Jul 02 23:10:56.815906 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:56.815884    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00f9bd621d84080c84f579c3e648158f.slice/crio-9d14fdfb212ed166511b3437850c967e1ed4665c2f63e0f85940757b321c0e3d WatchSource:0}: Error finding container 9d14fdfb212ed166511b3437850c967e1ed4665c2f63e0f85940757b321c0e3d: Status 404 returned error can't find the container with id 9d14fdfb212ed166511b3437850c967e1ed4665c2f63e0f85940757b321c0e3d
Jul 02 23:10:56.820814 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:56.820796    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 23:10:57.071442 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.071347    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 23:10:57.130964 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.130925    2580 apiserver.go:52] "Watching apiserver"
Jul 02 23:10:57.139918 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.139759    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 02 23:10:57.140338 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.140310    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-g5qvg","kube-system/global-pull-secret-syncer-vdlx9","kube-system/konnectivity-agent-thnfg","kube-system/kube-apiserver-proxy-ip-10-0-140-52.ec2.internal","openshift-multus/multus-additional-cni-plugins-5jsxq","openshift-multus/multus-dvdgd","openshift-multus/network-metrics-daemon-hjstr","openshift-network-diagnostics/network-check-target-l9lgl","openshift-ovn-kubernetes/ovnkube-node-9xtgc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj","openshift-cluster-node-tuning-operator/tuned-m8pqn","openshift-dns/node-resolver-vhqb2","openshift-image-registry/node-ca-54qq8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal"]
Jul 02 23:10:57.141926 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.141905    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-thnfg"
Jul 02 23:10:57.143353 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.143332    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:10:57.143458 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.143404    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:10:57.144856 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.144625    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g5qvg"
Jul 02 23:10:57.144856 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.144808    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 02 23:10:57.145013 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.144941    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 02 23:10:57.145013 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.145006    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fpnsn\""
Jul 02 23:10:57.146709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.146299    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.148703 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.148050    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 02 23:10:57.148703 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.148606    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tscx7\""
Jul 02 23:10:57.148844 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.148799    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 02 23:10:57.148894 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.148878    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 02 23:10:57.149181 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.149081    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 02 23:10:57.149181 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.149159    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 02 23:10:57.150297 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.149327    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 02 23:10:57.150297 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.149592    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.150297 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.149743    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 02 23:10:57.150297 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.149933    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 02 23:10:57.151236 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.150687    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 02 23:10:57.151236 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.150841    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s46rg\""
Jul 02 23:10:57.152881 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.152860    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 23:10:57.153139 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.153100    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 02 23:10:57.153250 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.153196    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h5ttn\""
Jul 02 23:10:57.153460 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.153443    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:10:57.153572 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.153527    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:10:57.155115 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.155095    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:10:57.155227 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.155175    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:10:57.157366 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.157347    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.158962 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.158696    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.159821 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.159661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-multus-conf-dir\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.159821 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.159697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eed7310e-b623-4eca-91a1-c4c6bc3df056-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.159821 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.159723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-cnibin\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.159821 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.159746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-var-lib-cni-bin\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.159821 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.159772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-run-multus-certs\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.159821 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.159795    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5b81881b-bb30-4123-bfdc-e23c11f55ba1-iptables-alerter-script\") pod \"iptables-alerter-g5qvg\" (UID: \"5b81881b-bb30-4123-bfdc-e23c11f55ba1\") " pod="openshift-network-operator/iptables-alerter-g5qvg"
Jul 02 23:10:57.159821 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.159818    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-etc-kubernetes\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.160301 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.159838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b81881b-bb30-4123-bfdc-e23c11f55ba1-host-slash\") pod \"iptables-alerter-g5qvg\" (UID: \"5b81881b-bb30-4123-bfdc-e23c11f55ba1\") " pod="openshift-network-operator/iptables-alerter-g5qvg"
Jul 02 23:10:57.160301 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.159879    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-multus-socket-dir-parent\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.160301 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.159902    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv9jm\" (UniqueName: \"kubernetes.io/projected/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-kube-api-access-vv9jm\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.160301 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.159926    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-kubelet-config\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:10:57.160301 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.159951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:10:57.160301 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.159975    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-var-lib-cni-multus\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.160301 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.160001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eed7310e-b623-4eca-91a1-c4c6bc3df056-cnibin\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.160301 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.160102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eed7310e-b623-4eca-91a1-c4c6bc3df056-os-release\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.160301 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.160154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-system-cni-dir\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.160301 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.160178    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-run-k8s-cni-cncf-io\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.160764 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.160532    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 02 23:10:57.160764 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.160557    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 02 23:10:57.160764 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.160688    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 02 23:10:57.160764 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.160762    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 02 23:10:57.160974 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.160214    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlq24\" (UniqueName: \"kubernetes.io/projected/d7a0204c-3f9f-4f10-ace5-def5e62925dd-kube-api-access-rlq24\") pod \"network-metrics-daemon-hjstr\" (UID: \"d7a0204c-3f9f-4f10-ace5-def5e62925dd\") " pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:10:57.161070 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.160995    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eed7310e-b623-4eca-91a1-c4c6bc3df056-system-cni-dir\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.161070 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-os-release\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.161070 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161044    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-run-netns\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.161243 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-var-lib-kubelet\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.161243 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161115    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/220907ed-e9d9-4b0c-9b89-7be9fdd4552e-agent-certs\") pod \"konnectivity-agent-thnfg\" (UID: \"220907ed-e9d9-4b0c-9b89-7be9fdd4552e\") " pod="kube-system/konnectivity-agent-thnfg"
Jul 02 23:10:57.161243 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eed7310e-b623-4eca-91a1-c4c6bc3df056-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.161243 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161178    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l96nq\" (UniqueName: \"kubernetes.io/projected/eed7310e-b623-4eca-91a1-c4c6bc3df056-kube-api-access-l96nq\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.161243 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161209    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-multus-cni-dir\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.161243 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-dbus\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:10:57.161523 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vb7t\" (UniqueName: \"kubernetes.io/projected/5b81881b-bb30-4123-bfdc-e23c11f55ba1-kube-api-access-8vb7t\") pod \"iptables-alerter-g5qvg\" (UID: \"5b81881b-bb30-4123-bfdc-e23c11f55ba1\") " pod="openshift-network-operator/iptables-alerter-g5qvg"
Jul 02 23:10:57.161523 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161295    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eed7310e-b623-4eca-91a1-c4c6bc3df056-cni-binary-copy\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.161523 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eed7310e-b623-4eca-91a1-c4c6bc3df056-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.161523 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-cni-binary-copy\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.161523 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161375    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-multus-daemon-config\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.161523 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161398    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/220907ed-e9d9-4b0c-9b89-7be9fdd4552e-konnectivity-ca\") pod \"konnectivity-agent-thnfg\" (UID: \"220907ed-e9d9-4b0c-9b89-7be9fdd4552e\") " pod="kube-system/konnectivity-agent-thnfg"
Jul 02 23:10:57.161523 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161422    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs\") pod \"network-metrics-daemon-hjstr\" (UID: \"d7a0204c-3f9f-4f10-ace5-def5e62925dd\") " pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:10:57.161523 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161444    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-hostroot\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.161523 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.161451    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.162429 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.162408    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-n9hnj\""
Jul 02 23:10:57.162658 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.162639    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 02 23:10:57.162761 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.162682    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8m6js\""
Jul 02 23:10:57.162828 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.162536    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 02 23:10:57.162970 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.162479    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 02 23:10:57.163053 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.163041    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 02 23:10:57.163186 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.163168    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 02 23:10:57.163265 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.163183    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 02 23:10:57.164351 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.164325    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 02 23:10:57.164447 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.164329    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r6spk\""
Jul 02 23:10:57.164447 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.164405    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 02 23:10:57.164567 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.164559    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vhqb2"
Jul 02 23:10:57.165741 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.165588    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-54qq8"
Jul 02 23:10:57.166989 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.166968    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 02 23:10:57.167868 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.167456    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4v6fh\""
Jul 02 23:10:57.167868 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.167558    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 02 23:10:57.168696 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.168494    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 02 23:10:57.168696 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.168542    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 02 23:10:57.168696 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.168614    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 02 23:10:57.168696 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.168614    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w4clk\""
Jul 02 23:10:57.184381 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.183618    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pzk4n"
Jul 02 23:10:57.191981 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.191784    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pzk4n"
Jul 02 23:10:57.251642 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.251615    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 02 23:10:57.262575 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262542    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.262575 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-modprobe-d\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.262786 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262600    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-kubernetes\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.262786 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262618    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-host\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.262786 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262642    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/209988dd-4347-4ac0-ba98-2afa7b5ce94b-tmp\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.262786 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262670    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5b81881b-bb30-4123-bfdc-e23c11f55ba1-iptables-alerter-script\") pod \"iptables-alerter-g5qvg\" (UID: \"5b81881b-bb30-4123-bfdc-e23c11f55ba1\") " pod="openshift-network-operator/iptables-alerter-g5qvg"
Jul 02 23:10:57.262786 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262695    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b22jg\" (UniqueName: \"kubernetes.io/projected/cd52fd45-f3e3-47fb-9725-c1fd501e104b-kube-api-access-b22jg\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.262786 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262716    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-sys\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.262786 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262740    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-multus-socket-dir-parent\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.262786 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262763    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-socket-dir\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.262786 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-etc-selinux\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.263274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262807    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-var-lib-openvswitch\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.263274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-env-overrides\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.263274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262852    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-run\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.263274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-device-dir\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.263274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262887    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-multus-socket-dir-parent\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.263274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-systemd-units\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.263274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262955    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-run-ovn\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.263274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.262984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-tuned\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.263274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263007    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-run-netns\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.263274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263037    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlq24\" (UniqueName: \"kubernetes.io/projected/d7a0204c-3f9f-4f10-ace5-def5e62925dd-kube-api-access-rlq24\") pod \"network-metrics-daemon-hjstr\" (UID: \"d7a0204c-3f9f-4f10-ace5-def5e62925dd\") " pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:10:57.263274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.263274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-os-release\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.263274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-run-netns\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.263274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-var-lib-kubelet\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.263972 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/220907ed-e9d9-4b0c-9b89-7be9fdd4552e-agent-certs\") pod \"konnectivity-agent-thnfg\" (UID: \"220907ed-e9d9-4b0c-9b89-7be9fdd4552e\") " pod="kube-system/konnectivity-agent-thnfg"
Jul 02 23:10:57.263972 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263198    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-cni-netd\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.263972 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eed7310e-b623-4eca-91a1-c4c6bc3df056-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.263972 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l96nq\" (UniqueName: \"kubernetes.io/projected/eed7310e-b623-4eca-91a1-c4c6bc3df056-kube-api-access-l96nq\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.263972 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263274    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-multus-cni-dir\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.263972 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263297    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-kubelet\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.263972 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-etc-openvswitch\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.263972 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263341    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-run-openvswitch\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.263972 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263351    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5b81881b-bb30-4123-bfdc-e23c11f55ba1-iptables-alerter-script\") pod \"iptables-alerter-g5qvg\" (UID: \"5b81881b-bb30-4123-bfdc-e23c11f55ba1\") " pod="openshift-network-operator/iptables-alerter-g5qvg"
Jul 02 23:10:57.263972 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-log-socket\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.263972 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-ovnkube-config\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.263972 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vb7t\" (UniqueName: \"kubernetes.io/projected/5b81881b-bb30-4123-bfdc-e23c11f55ba1-kube-api-access-8vb7t\") pod \"iptables-alerter-g5qvg\" (UID: \"5b81881b-bb30-4123-bfdc-e23c11f55ba1\") " pod="openshift-network-operator/iptables-alerter-g5qvg"
Jul 02 23:10:57.263972 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263474    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-os-release\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.263972 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eed7310e-b623-4eca-91a1-c4c6bc3df056-cni-binary-copy\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.264664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-cni-binary-copy\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.264664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263545    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-multus-daemon-config\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.264664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263570    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/220907ed-e9d9-4b0c-9b89-7be9fdd4552e-konnectivity-ca\") pod \"konnectivity-agent-thnfg\" (UID: \"220907ed-e9d9-4b0c-9b89-7be9fdd4552e\") " pod="kube-system/konnectivity-agent-thnfg"
Jul 02 23:10:57.264664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-run-ovn-kubernetes\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.264664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-ovnkube-script-lib\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.264664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263647    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nlld\" (UniqueName: \"kubernetes.io/projected/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-kube-api-access-9nlld\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.264664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263649    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 02 23:10:57.264664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-hostroot\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.264664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-multus-conf-dir\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.264664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263764    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-hostroot\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.264664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263817    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-run-netns\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.264664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-cnibin\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.264664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263890    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-var-lib-cni-bin\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.264664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b9e5b05d-952a-4a24-8221-9c64bfa92ec8-serviceca\") pod \"node-ca-54qq8\" (UID: \"b9e5b05d-952a-4a24-8221-9c64bfa92ec8\") " pod="openshift-image-registry/node-ca-54qq8"
Jul 02 23:10:57.264664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263942    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-sysctl-conf\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.265387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.263967    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcwz8\" (UniqueName: \"kubernetes.io/projected/209988dd-4347-4ac0-ba98-2afa7b5ce94b-kube-api-access-vcwz8\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.265387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264042    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-etc-kubernetes\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.265387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264072    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72a6430a-6d3b-489d-afb8-465243b1bf36-tmp-dir\") pod \"node-resolver-vhqb2\" (UID: \"72a6430a-6d3b-489d-afb8-465243b1bf36\") " pod="openshift-dns/node-resolver-vhqb2"
Jul 02 23:10:57.265387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264077    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-multus-cni-dir\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.265387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264077    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eed7310e-b623-4eca-91a1-c4c6bc3df056-cni-binary-copy\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.265387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264099    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-sysctl-d\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.265387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264117    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-var-lib-cni-bin\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.265387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264146    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-lib-modules\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.265387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264171    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn5kd\" (UniqueName: \"kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd\") pod \"network-check-target-l9lgl\" (UID: \"8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd\") " pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:10:57.265387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b81881b-bb30-4123-bfdc-e23c11f55ba1-host-slash\") pod \"iptables-alerter-g5qvg\" (UID: \"5b81881b-bb30-4123-bfdc-e23c11f55ba1\") " pod="openshift-network-operator/iptables-alerter-g5qvg"
Jul 02 23:10:57.265387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv9jm\" (UniqueName: \"kubernetes.io/projected/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-kube-api-access-vv9jm\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.265387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264251    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9e5b05d-952a-4a24-8221-9c64bfa92ec8-host\") pod \"node-ca-54qq8\" (UID: \"b9e5b05d-952a-4a24-8221-9c64bfa92ec8\") " pod="openshift-image-registry/node-ca-54qq8"
Jul 02 23:10:57.265387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264275    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-slash\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.265387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264280    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/220907ed-e9d9-4b0c-9b89-7be9fdd4552e-konnectivity-ca\") pod \"konnectivity-agent-thnfg\" (UID: \"220907ed-e9d9-4b0c-9b89-7be9fdd4552e\") " pod="kube-system/konnectivity-agent-thnfg"
Jul 02 23:10:57.266081 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264301    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-run-systemd\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.266081 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264329    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-multus-conf-dir\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.266081 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-kubelet-config\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:10:57.266081 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264399    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-var-lib-kubelet\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.266081 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264420    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:10:57.266081 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5hmd\" (UniqueName: \"kubernetes.io/projected/b9e5b05d-952a-4a24-8221-9c64bfa92ec8-kube-api-access-t5hmd\") pod \"node-ca-54qq8\" (UID: \"b9e5b05d-952a-4a24-8221-9c64bfa92ec8\") " pod="openshift-image-registry/node-ca-54qq8"
Jul 02 23:10:57.266081 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264490    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-var-lib-kubelet\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.266081 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-var-lib-cni-multus\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.266081 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264545    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eed7310e-b623-4eca-91a1-c4c6bc3df056-cnibin\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.266081 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264569    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eed7310e-b623-4eca-91a1-c4c6bc3df056-os-release\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.266081 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-system-cni-dir\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.266081 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264606    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eed7310e-b623-4eca-91a1-c4c6bc3df056-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.266081 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-run-k8s-cni-cncf-io\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.266081 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264634    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-multus-daemon-config\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.266728 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264647    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-etc-kubernetes\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.266728 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264710    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-cnibin\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.266728 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264737    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-node-log\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.266728 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264752    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-kubelet-config\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:10:57.266728 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-cni-bin\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.266728 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264787    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-run-k8s-cni-cncf-io\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.266728 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eed7310e-b623-4eca-91a1-c4c6bc3df056-system-cni-dir\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.266728 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264821    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-var-lib-cni-multus\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.266728 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264823    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-systemd\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.266728 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264852    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-dbus\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:10:57.266728 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264855    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eed7310e-b623-4eca-91a1-c4c6bc3df056-cnibin\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.266728 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.264936    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 23:10:57.266728 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.264940    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-dbus\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:10:57.266728 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.264998    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret podName:e1d3f804-fd8f-43fa-81ab-82a96cc020e0 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:57.764976331 +0000 UTC m=+3.018766329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret") pod "global-pull-secret-syncer-vdlx9" (UID: "e1d3f804-fd8f-43fa-81ab-82a96cc020e0") : object "kube-system"/"original-pull-secret" not registered
Jul 02 23:10:57.267412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265051    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-system-cni-dir\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.267412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265105    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eed7310e-b623-4eca-91a1-c4c6bc3df056-os-release\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.267412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265150    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-sys-fs\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.267412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265179    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/72a6430a-6d3b-489d-afb8-465243b1bf36-hosts-file\") pod \"node-resolver-vhqb2\" (UID: \"72a6430a-6d3b-489d-afb8-465243b1bf36\") " pod="openshift-dns/node-resolver-vhqb2"
Jul 02 23:10:57.267412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265209    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxnt9\" (UniqueName: \"kubernetes.io/projected/72a6430a-6d3b-489d-afb8-465243b1bf36-kube-api-access-pxnt9\") pod \"node-resolver-vhqb2\" (UID: \"72a6430a-6d3b-489d-afb8-465243b1bf36\") " pod="openshift-dns/node-resolver-vhqb2"
Jul 02 23:10:57.267412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265212    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b81881b-bb30-4123-bfdc-e23c11f55ba1-host-slash\") pod \"iptables-alerter-g5qvg\" (UID: \"5b81881b-bb30-4123-bfdc-e23c11f55ba1\") " pod="openshift-network-operator/iptables-alerter-g5qvg"
Jul 02 23:10:57.267412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eed7310e-b623-4eca-91a1-c4c6bc3df056-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.267412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265269    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eed7310e-b623-4eca-91a1-c4c6bc3df056-system-cni-dir\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.267412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs\") pod \"network-metrics-daemon-hjstr\" (UID: \"d7a0204c-3f9f-4f10-ace5-def5e62925dd\") " pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:10:57.267412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-ovn-node-metrics-cert\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.267412 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.265423    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 23:10:57.267412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-sysconfig\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.267412 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.265523    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs podName:d7a0204c-3f9f-4f10-ace5-def5e62925dd nodeName:}" failed. No retries permitted until 2026-07-02 23:10:57.765501984 +0000 UTC m=+3.019291964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs") pod "network-metrics-daemon-hjstr" (UID: "d7a0204c-3f9f-4f10-ace5-def5e62925dd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 23:10:57.267412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265496    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eed7310e-b623-4eca-91a1-c4c6bc3df056-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.268114 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265556    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eed7310e-b623-4eca-91a1-c4c6bc3df056-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.268114 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265599    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-run-multus-certs\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.268114 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265645    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-host-run-multus-certs\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.268114 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265703    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-registration-dir\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.268114 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.265786    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-cni-binary-copy\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.268114 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.266032    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eed7310e-b623-4eca-91a1-c4c6bc3df056-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.270554 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.270499    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/220907ed-e9d9-4b0c-9b89-7be9fdd4552e-agent-certs\") pod \"konnectivity-agent-thnfg\" (UID: \"220907ed-e9d9-4b0c-9b89-7be9fdd4552e\") " pod="kube-system/konnectivity-agent-thnfg"
Jul 02 23:10:57.276101 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.275997    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlq24\" (UniqueName: \"kubernetes.io/projected/d7a0204c-3f9f-4f10-ace5-def5e62925dd-kube-api-access-rlq24\") pod \"network-metrics-daemon-hjstr\" (UID: \"d7a0204c-3f9f-4f10-ace5-def5e62925dd\") " pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:10:57.276827 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.276804    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vb7t\" (UniqueName: \"kubernetes.io/projected/5b81881b-bb30-4123-bfdc-e23c11f55ba1-kube-api-access-8vb7t\") pod \"iptables-alerter-g5qvg\" (UID: \"5b81881b-bb30-4123-bfdc-e23c11f55ba1\") " pod="openshift-network-operator/iptables-alerter-g5qvg"
Jul 02 23:10:57.276940 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.276838    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv9jm\" (UniqueName: \"kubernetes.io/projected/c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7-kube-api-access-vv9jm\") pod \"multus-dvdgd\" (UID: \"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7\") " pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.277725 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.277685    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l96nq\" (UniqueName: \"kubernetes.io/projected/eed7310e-b623-4eca-91a1-c4c6bc3df056-kube-api-access-l96nq\") pod \"multus-additional-cni-plugins-5jsxq\" (UID: \"eed7310e-b623-4eca-91a1-c4c6bc3df056\") " pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.281675 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.281627    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal" event={"ID":"00f9bd621d84080c84f579c3e648158f","Type":"ContainerStarted","Data":"9d14fdfb212ed166511b3437850c967e1ed4665c2f63e0f85940757b321c0e3d"}
Jul 02 23:10:57.283473 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.283450    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-52.ec2.internal" event={"ID":"5a709290dff23a904b036c05f5cc09e5","Type":"ContainerStarted","Data":"a3121edbbf8f4c968e9f14aa6738e05fb7b9bb62e3b74400be6fe9657255e7fd"}
Jul 02 23:10:57.367895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366322    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-node-log\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.367895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-cni-bin\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.367895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-systemd\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.367895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-sys-fs\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.367895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366435    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-node-log\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.367895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/72a6430a-6d3b-489d-afb8-465243b1bf36-hosts-file\") pod \"node-resolver-vhqb2\" (UID: \"72a6430a-6d3b-489d-afb8-465243b1bf36\") " pod="openshift-dns/node-resolver-vhqb2"
Jul 02 23:10:57.367895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxnt9\" (UniqueName: \"kubernetes.io/projected/72a6430a-6d3b-489d-afb8-465243b1bf36-kube-api-access-pxnt9\") pod \"node-resolver-vhqb2\" (UID: \"72a6430a-6d3b-489d-afb8-465243b1bf36\") " pod="openshift-dns/node-resolver-vhqb2"
Jul 02 23:10:57.367895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-ovn-node-metrics-cert\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.367895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366516    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/72a6430a-6d3b-489d-afb8-465243b1bf36-hosts-file\") pod \"node-resolver-vhqb2\" (UID: \"72a6430a-6d3b-489d-afb8-465243b1bf36\") " pod="openshift-dns/node-resolver-vhqb2"
Jul 02 23:10:57.367895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-sysconfig\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.367895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366565    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-systemd\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.367895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366600    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-cni-bin\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.367895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366669    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-sys-fs\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.367895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366727    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-registration-dir\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.367895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366733    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-sysconfig\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.368788 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.368788 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-modprobe-d\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.368788 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366805    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-kubernetes\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.368788 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366828    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-host\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.368788 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/209988dd-4347-4ac0-ba98-2afa7b5ce94b-tmp\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.368788 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b22jg\" (UniqueName: \"kubernetes.io/projected/cd52fd45-f3e3-47fb-9725-c1fd501e104b-kube-api-access-b22jg\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.368788 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366898    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-sys\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.368788 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-socket-dir\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.368788 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366957    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-modprobe-d\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.368788 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366976    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-etc-selinux\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.368788 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.366999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-var-lib-openvswitch\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.368788 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367019    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-registration-dir\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.368788 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367025    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-env-overrides\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.368788 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367056    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-run\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.369487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367083    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-device-dir\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.369487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-systemd-units\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.369487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-run-ovn\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.369487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367172    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-tuned\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.369487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367190    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-device-dir\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.369487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-run-netns\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.369487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367234    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-run-netns\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.369487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.369487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367273    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-cni-netd\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.369487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367279    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.369487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367302    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-kubelet\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.369487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367327    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-run\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.369487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367364    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-etc-openvswitch\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.369487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367331    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-etc-openvswitch\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.369487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367386    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-systemd-units\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.370253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-run-openvswitch\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.370253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367426    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-run-ovn\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.370253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367143    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.370253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367484    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-log-socket\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.370253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-ovnkube-config\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.370253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367532    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-kubernetes\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.370253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-run-ovn-kubernetes\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.370253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-ovnkube-script-lib\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.370253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nlld\" (UniqueName: \"kubernetes.io/projected/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-kube-api-access-9nlld\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.370253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367616    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b9e5b05d-952a-4a24-8221-9c64bfa92ec8-serviceca\") pod \"node-ca-54qq8\" (UID: \"b9e5b05d-952a-4a24-8221-9c64bfa92ec8\") " pod="openshift-image-registry/node-ca-54qq8"
Jul 02 23:10:57.370253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-sysctl-conf\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.370253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcwz8\" (UniqueName: \"kubernetes.io/projected/209988dd-4347-4ac0-ba98-2afa7b5ce94b-kube-api-access-vcwz8\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.370253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367695    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72a6430a-6d3b-489d-afb8-465243b1bf36-tmp-dir\") pod \"node-resolver-vhqb2\" (UID: \"72a6430a-6d3b-489d-afb8-465243b1bf36\") " pod="openshift-dns/node-resolver-vhqb2"
Jul 02 23:10:57.370253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367712    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-run-ovn-kubernetes\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.370816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-sysctl-d\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.370816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367765    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-lib-modules\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.370816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5kd\" (UniqueName: \"kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd\") pod \"network-check-target-l9lgl\" (UID: \"8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd\") " pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:10:57.370816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9e5b05d-952a-4a24-8221-9c64bfa92ec8-host\") pod \"node-ca-54qq8\" (UID: \"b9e5b05d-952a-4a24-8221-9c64bfa92ec8\") " pod="openshift-image-registry/node-ca-54qq8"
Jul 02 23:10:57.370816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367833    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-sysctl-d\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.370816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-slash\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.370816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367870    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-socket-dir\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.370816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367911    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-run-systemd\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.370816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367483    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-env-overrides\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.370816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367944    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-log-socket\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.370816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.368210    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-sys\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.370816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-run-systemd\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.370816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.368268    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5hmd\" (UniqueName: \"kubernetes.io/projected/b9e5b05d-952a-4a24-8221-9c64bfa92ec8-kube-api-access-t5hmd\") pod \"node-ca-54qq8\" (UID: \"b9e5b05d-952a-4a24-8221-9c64bfa92ec8\") " pod="openshift-image-registry/node-ca-54qq8"
Jul 02 23:10:57.370816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.368294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-var-lib-kubelet\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.370816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.368375    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-var-lib-kubelet\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.371561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367912    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-run-openvswitch\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.371561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.368411    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-ovnkube-config\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.371561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.368427    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-ovnkube-script-lib\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.371561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.368529    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-lib-modules\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.371561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.368606    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-kubelet\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.371561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.367588    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-host\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.371561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.368647    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-cni-netd\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.371561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.368721    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9e5b05d-952a-4a24-8221-9c64bfa92ec8-host\") pod \"node-ca-54qq8\" (UID: \"b9e5b05d-952a-4a24-8221-9c64bfa92ec8\") " pod="openshift-image-registry/node-ca-54qq8"
Jul 02 23:10:57.371561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.368765    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-host-slash\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.371561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.368885    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-sysctl-conf\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.371561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.368909    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cd52fd45-f3e3-47fb-9725-c1fd501e104b-etc-selinux\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.371561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.368939    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-var-lib-openvswitch\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.371561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.369037    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b9e5b05d-952a-4a24-8221-9c64bfa92ec8-serviceca\") pod \"node-ca-54qq8\" (UID: \"b9e5b05d-952a-4a24-8221-9c64bfa92ec8\") " pod="openshift-image-registry/node-ca-54qq8"
Jul 02 23:10:57.371561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.369192    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72a6430a-6d3b-489d-afb8-465243b1bf36-tmp-dir\") pod \"node-resolver-vhqb2\" (UID: \"72a6430a-6d3b-489d-afb8-465243b1bf36\") " pod="openshift-dns/node-resolver-vhqb2"
Jul 02 23:10:57.371561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.370240    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-ovn-node-metrics-cert\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.372353 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.371147    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/209988dd-4347-4ac0-ba98-2afa7b5ce94b-tmp\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.372353 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.371456    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/209988dd-4347-4ac0-ba98-2afa7b5ce94b-etc-tuned\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.376203 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.376072    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxnt9\" (UniqueName: \"kubernetes.io/projected/72a6430a-6d3b-489d-afb8-465243b1bf36-kube-api-access-pxnt9\") pod \"node-resolver-vhqb2\" (UID: \"72a6430a-6d3b-489d-afb8-465243b1bf36\") " pod="openshift-dns/node-resolver-vhqb2"
Jul 02 23:10:57.424271 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.424241    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 23:10:57.424271 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.424276    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 23:10:57.424271 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.424291    2580 projected.go:196] Error preparing data for projected volume kube-api-access-cn5kd for pod openshift-network-diagnostics/network-check-target-l9lgl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 23:10:57.424555 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.424354    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd podName:8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd nodeName:}" failed. No retries permitted until 2026-07-02 23:10:57.92433629 +0000 UTC m=+3.178126269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cn5kd" (UniqueName: "kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd") pod "network-check-target-l9lgl" (UID: "8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 23:10:57.426706 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.426666    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b22jg\" (UniqueName: \"kubernetes.io/projected/cd52fd45-f3e3-47fb-9725-c1fd501e104b-kube-api-access-b22jg\") pod \"aws-ebs-csi-driver-node-zmbnj\" (UID: \"cd52fd45-f3e3-47fb-9725-c1fd501e104b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.428499 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.428447    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nlld\" (UniqueName: \"kubernetes.io/projected/da9201f5-a1c4-4e71-b7f8-beb4ef769b4f-kube-api-access-9nlld\") pod \"ovnkube-node-9xtgc\" (UID: \"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.428499 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.428467    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcwz8\" (UniqueName: \"kubernetes.io/projected/209988dd-4347-4ac0-ba98-2afa7b5ce94b-kube-api-access-vcwz8\") pod \"tuned-m8pqn\" (UID: \"209988dd-4347-4ac0-ba98-2afa7b5ce94b\") " pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.429080 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.429054    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5hmd\" (UniqueName: \"kubernetes.io/projected/b9e5b05d-952a-4a24-8221-9c64bfa92ec8-kube-api-access-t5hmd\") pod \"node-ca-54qq8\" (UID: \"b9e5b05d-952a-4a24-8221-9c64bfa92ec8\") " pod="openshift-image-registry/node-ca-54qq8"
Jul 02 23:10:57.455451 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.455362    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-thnfg"
Jul 02 23:10:57.464233 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.464210    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5jsxq"
Jul 02 23:10:57.465896 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:57.465837    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod220907ed_e9d9_4b0c_9b89_7be9fdd4552e.slice/crio-7574156c648409692a732b7079bda56076557bcdeced0614fb4dc4fc863fe0b6 WatchSource:0}: Error finding container 7574156c648409692a732b7079bda56076557bcdeced0614fb4dc4fc863fe0b6: Status 404 returned error can't find the container with id 7574156c648409692a732b7079bda56076557bcdeced0614fb4dc4fc863fe0b6
Jul 02 23:10:57.473959 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.473938    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g5qvg"
Jul 02 23:10:57.475916 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:57.475771    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeed7310e_b623_4eca_91a1_c4c6bc3df056.slice/crio-9a928b1c001b965456084419d616facdb5fea52fb524a3f4b78bd1a9e22d3365 WatchSource:0}: Error finding container 9a928b1c001b965456084419d616facdb5fea52fb524a3f4b78bd1a9e22d3365: Status 404 returned error can't find the container with id 9a928b1c001b965456084419d616facdb5fea52fb524a3f4b78bd1a9e22d3365
Jul 02 23:10:57.481367 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.481217    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dvdgd"
Jul 02 23:10:57.483455 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:57.483392    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b81881b_bb30_4123_bfdc_e23c11f55ba1.slice/crio-477fbb87694b2b4e5e9f8e034259b53ffc0651dc1f46a0827034c4b2414969b9 WatchSource:0}: Error finding container 477fbb87694b2b4e5e9f8e034259b53ffc0651dc1f46a0827034c4b2414969b9: Status 404 returned error can't find the container with id 477fbb87694b2b4e5e9f8e034259b53ffc0651dc1f46a0827034c4b2414969b9
Jul 02 23:10:57.489054 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.489033    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:10:57.489812 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:57.489785    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0c3ed8f_cc22_4981_8a71_fda36e2b9cf7.slice/crio-585bb48e91da4bcc6e9b5f4d7aa33aa5ec35ac3b3c4a63c8cf351ded9e4d5630 WatchSource:0}: Error finding container 585bb48e91da4bcc6e9b5f4d7aa33aa5ec35ac3b3c4a63c8cf351ded9e4d5630: Status 404 returned error can't find the container with id 585bb48e91da4bcc6e9b5f4d7aa33aa5ec35ac3b3c4a63c8cf351ded9e4d5630
Jul 02 23:10:57.497242 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.497222    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj"
Jul 02 23:10:57.497950 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:57.497928    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda9201f5_a1c4_4e71_b7f8_beb4ef769b4f.slice/crio-05d4d199a631301c2b192fb5a25f533e3e3ab48fc8c660e5e7d60a30612855c5 WatchSource:0}: Error finding container 05d4d199a631301c2b192fb5a25f533e3e3ab48fc8c660e5e7d60a30612855c5: Status 404 returned error can't find the container with id 05d4d199a631301c2b192fb5a25f533e3e3ab48fc8c660e5e7d60a30612855c5
Jul 02 23:10:57.505537 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.505518    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m8pqn"
Jul 02 23:10:57.515602 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.515443    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vhqb2"
Jul 02 23:10:57.515602 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:57.515550    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod209988dd_4347_4ac0_ba98_2afa7b5ce94b.slice/crio-2df454e7c93721a1ca1108d5a6ae4f083e943048e4280e5a8ba3fecb9748cb15 WatchSource:0}: Error finding container 2df454e7c93721a1ca1108d5a6ae4f083e943048e4280e5a8ba3fecb9748cb15: Status 404 returned error can't find the container with id 2df454e7c93721a1ca1108d5a6ae4f083e943048e4280e5a8ba3fecb9748cb15
Jul 02 23:10:57.521368 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.521210    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-54qq8"
Jul 02 23:10:57.523529 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:57.523400    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72a6430a_6d3b_489d_afb8_465243b1bf36.slice/crio-a6573b87ac3a7f9f75917739b1a847c9b9d7b344a36807e4e06546131f504320 WatchSource:0}: Error finding container a6573b87ac3a7f9f75917739b1a847c9b9d7b344a36807e4e06546131f504320: Status 404 returned error can't find the container with id a6573b87ac3a7f9f75917739b1a847c9b9d7b344a36807e4e06546131f504320
Jul 02 23:10:57.530324 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:10:57.530300    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9e5b05d_952a_4a24_8221_9c64bfa92ec8.slice/crio-e5f91d29f94d01f2926b083162ed18202c40f9eb5a70dfd35e00ec0643c11508 WatchSource:0}: Error finding container e5f91d29f94d01f2926b083162ed18202c40f9eb5a70dfd35e00ec0643c11508: Status 404 returned error can't find the container with id e5f91d29f94d01f2926b083162ed18202c40f9eb5a70dfd35e00ec0643c11508
Jul 02 23:10:57.771064 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.770973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:10:57.771064 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.771032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs\") pod \"network-metrics-daemon-hjstr\" (UID: \"d7a0204c-3f9f-4f10-ace5-def5e62925dd\") " pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:10:57.771279 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.771132    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 23:10:57.771279 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.771166    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 23:10:57.771279 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.771206    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret podName:e1d3f804-fd8f-43fa-81ab-82a96cc020e0 nodeName:}" failed. No retries permitted until 2026-07-02 23:10:58.771188602 +0000 UTC m=+4.024978600 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret") pod "global-pull-secret-syncer-vdlx9" (UID: "e1d3f804-fd8f-43fa-81ab-82a96cc020e0") : object "kube-system"/"original-pull-secret" not registered
Jul 02 23:10:57.771279 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.771228    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs podName:d7a0204c-3f9f-4f10-ace5-def5e62925dd nodeName:}" failed. No retries permitted until 2026-07-02 23:10:58.771218348 +0000 UTC m=+4.025008324 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs") pod "network-metrics-daemon-hjstr" (UID: "d7a0204c-3f9f-4f10-ace5-def5e62925dd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 23:10:57.972157 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:57.972110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5kd\" (UniqueName: \"kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd\") pod \"network-check-target-l9lgl\" (UID: \"8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd\") " pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:10:57.972302 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.972222    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 23:10:57.972302 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.972240    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 23:10:57.972302 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.972249    2580 projected.go:196] Error preparing data for projected volume kube-api-access-cn5kd for pod openshift-network-diagnostics/network-check-target-l9lgl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 23:10:57.972302 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:57.972296    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd podName:8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd nodeName:}" failed. No retries permitted until 2026-07-02 23:10:58.972282065 +0000 UTC m=+4.226072041 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cn5kd" (UniqueName: "kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd") pod "network-check-target-l9lgl" (UID: "8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 23:10:58.194644 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.194494    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 23:05:57 +0000 UTC" deadline="2027-12-09 07:00:11.35248814 +0000 UTC"
Jul 02 23:10:58.194644 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.194577    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12583h49m13.15791534s"
Jul 02 23:10:58.277308 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.277238    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:10:58.277499 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:58.277372    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:10:58.287167 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.287110    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj" event={"ID":"cd52fd45-f3e3-47fb-9725-c1fd501e104b","Type":"ContainerStarted","Data":"23d18d4bce5d23815fb604aeec6f04433b48398ee2bb1d389ee359ca6b698d10"}
Jul 02 23:10:58.288601 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.288567    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc" event={"ID":"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f","Type":"ContainerStarted","Data":"05d4d199a631301c2b192fb5a25f533e3e3ab48fc8c660e5e7d60a30612855c5"}
Jul 02 23:10:58.289823 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.289782    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dvdgd" event={"ID":"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7","Type":"ContainerStarted","Data":"585bb48e91da4bcc6e9b5f4d7aa33aa5ec35ac3b3c4a63c8cf351ded9e4d5630"}
Jul 02 23:10:58.291405 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.291338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g5qvg" event={"ID":"5b81881b-bb30-4123-bfdc-e23c11f55ba1","Type":"ContainerStarted","Data":"477fbb87694b2b4e5e9f8e034259b53ffc0651dc1f46a0827034c4b2414969b9"}
Jul 02 23:10:58.293881 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.293857    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jsxq" event={"ID":"eed7310e-b623-4eca-91a1-c4c6bc3df056","Type":"ContainerStarted","Data":"9a928b1c001b965456084419d616facdb5fea52fb524a3f4b78bd1a9e22d3365"}
Jul 02 23:10:58.295687 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.295664    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vhqb2" event={"ID":"72a6430a-6d3b-489d-afb8-465243b1bf36","Type":"ContainerStarted","Data":"a6573b87ac3a7f9f75917739b1a847c9b9d7b344a36807e4e06546131f504320"}
Jul 02 23:10:58.296956 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.296927    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m8pqn" event={"ID":"209988dd-4347-4ac0-ba98-2afa7b5ce94b","Type":"ContainerStarted","Data":"2df454e7c93721a1ca1108d5a6ae4f083e943048e4280e5a8ba3fecb9748cb15"}
Jul 02 23:10:58.298395 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.298366    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-thnfg" event={"ID":"220907ed-e9d9-4b0c-9b89-7be9fdd4552e","Type":"ContainerStarted","Data":"7574156c648409692a732b7079bda56076557bcdeced0614fb4dc4fc863fe0b6"}
Jul 02 23:10:58.299387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.299363    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-54qq8" event={"ID":"b9e5b05d-952a-4a24-8221-9c64bfa92ec8","Type":"ContainerStarted","Data":"e5f91d29f94d01f2926b083162ed18202c40f9eb5a70dfd35e00ec0643c11508"}
Jul 02 23:10:58.594627 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.594540    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 02 23:10:58.778602 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.778555    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:10:58.778791 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.778617    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs\") pod \"network-metrics-daemon-hjstr\" (UID: \"d7a0204c-3f9f-4f10-ace5-def5e62925dd\") " pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:10:58.778791 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:58.778737    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 23:10:58.778904 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:58.778846    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret podName:e1d3f804-fd8f-43fa-81ab-82a96cc020e0 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:00.778825712 +0000 UTC m=+6.032615692 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret") pod "global-pull-secret-syncer-vdlx9" (UID: "e1d3f804-fd8f-43fa-81ab-82a96cc020e0") : object "kube-system"/"original-pull-secret" not registered
Jul 02 23:10:58.778904 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:58.778741    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 23:10:58.779008 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:58.778942    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs podName:d7a0204c-3f9f-4f10-ace5-def5e62925dd nodeName:}" failed. No retries permitted until 2026-07-02 23:11:00.778930203 +0000 UTC m=+6.032720183 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs") pod "network-metrics-daemon-hjstr" (UID: "d7a0204c-3f9f-4f10-ace5-def5e62925dd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 23:10:58.980190 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:58.980093    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5kd\" (UniqueName: \"kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd\") pod \"network-check-target-l9lgl\" (UID: \"8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd\") " pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:10:58.980351 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:58.980294    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 23:10:58.980351 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:58.980321    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 23:10:58.980351 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:58.980334    2580 projected.go:196] Error preparing data for projected volume kube-api-access-cn5kd for pod openshift-network-diagnostics/network-check-target-l9lgl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 23:10:58.980531 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:58.980393    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd podName:8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd nodeName:}" failed. No retries permitted until 2026-07-02 23:11:00.980375052 +0000 UTC m=+6.234165043 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cn5kd" (UniqueName: "kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd") pod "network-check-target-l9lgl" (UID: "8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 23:10:59.195425 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:59.195380    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 23:05:57 +0000 UTC" deadline="2028-01-03 12:50:27.707546869 +0000 UTC"
Jul 02 23:10:59.195425 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:59.195424    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13189h39m28.512127235s"
Jul 02 23:10:59.279909 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:59.279827    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:10:59.280065 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:59.279963    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:10:59.280152 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:10:59.280065    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:10:59.280204 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:10:59.280168    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:00.278247 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:00.277499    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:00.278247 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:00.277852    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:00.326929 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:00.326182    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal" event={"ID":"00f9bd621d84080c84f579c3e648158f","Type":"ContainerStarted","Data":"91488040f981a4da24a1dc920833a387ef86ce7b0bc3ea1612ad06f304022f1e"}
Jul 02 23:11:00.334423 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:00.334385    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-52.ec2.internal" event={"ID":"5a709290dff23a904b036c05f5cc09e5","Type":"ContainerStarted","Data":"fabf561a91e155cf5f33ce3bcd3210550ea271c611d128e9fb82f3f6bb9b31b8"}
Jul 02 23:11:00.796739 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:00.796702    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:00.796933 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:00.796763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs\") pod \"network-metrics-daemon-hjstr\" (UID: \"d7a0204c-3f9f-4f10-ace5-def5e62925dd\") " pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:00.796933 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:00.796898    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 23:11:00.797061 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:00.796963    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs podName:d7a0204c-3f9f-4f10-ace5-def5e62925dd nodeName:}" failed. No retries permitted until 2026-07-02 23:11:04.796944405 +0000 UTC m=+10.050734398 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs") pod "network-metrics-daemon-hjstr" (UID: "d7a0204c-3f9f-4f10-ace5-def5e62925dd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 23:11:00.797380 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:00.797359    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 23:11:00.797481 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:00.797425    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret podName:e1d3f804-fd8f-43fa-81ab-82a96cc020e0 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:04.797402292 +0000 UTC m=+10.051192272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret") pod "global-pull-secret-syncer-vdlx9" (UID: "e1d3f804-fd8f-43fa-81ab-82a96cc020e0") : object "kube-system"/"original-pull-secret" not registered
Jul 02 23:11:00.998043 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:00.998010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5kd\" (UniqueName: \"kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd\") pod \"network-check-target-l9lgl\" (UID: \"8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd\") " pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:00.998270 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:00.998251    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 23:11:00.998343 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:00.998276    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 23:11:00.998343 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:00.998289    2580 projected.go:196] Error preparing data for projected volume kube-api-access-cn5kd for pod openshift-network-diagnostics/network-check-target-l9lgl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 23:11:00.998456 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:00.998345    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd podName:8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd nodeName:}" failed. No retries permitted until 2026-07-02 23:11:04.998326668 +0000 UTC m=+10.252116662 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cn5kd" (UniqueName: "kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd") pod "network-check-target-l9lgl" (UID: "8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 23:11:01.279490 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:01.279401    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:01.279993 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:01.279561    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:01.279993 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:01.279946    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:01.280135 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:01.280045    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:02.277298 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:02.277260    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:02.277477 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:02.277409    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:02.340939 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:02.340901    2580 generic.go:356] "Generic (PLEG): container finished" podID="00f9bd621d84080c84f579c3e648158f" containerID="91488040f981a4da24a1dc920833a387ef86ce7b0bc3ea1612ad06f304022f1e" exitCode=0
Jul 02 23:11:02.341431 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:02.340962    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal" event={"ID":"00f9bd621d84080c84f579c3e648158f","Type":"ContainerDied","Data":"91488040f981a4da24a1dc920833a387ef86ce7b0bc3ea1612ad06f304022f1e"}
Jul 02 23:11:02.360464 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:02.360359    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-52.ec2.internal" podStartSLOduration=6.360338284 podStartE2EDuration="6.360338284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:00.358345049 +0000 UTC m=+5.612135051" watchObservedRunningTime="2026-07-02 23:11:02.360338284 +0000 UTC m=+7.614128284"
Jul 02 23:11:03.277845 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:03.277410    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:03.277845 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:03.277564    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:03.278111 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:03.278095    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:03.278248 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:03.278212    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:04.277597 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:04.277500    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:04.278086 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:04.277638    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:04.831380 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:04.831332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:04.831609 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:04.831392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs\") pod \"network-metrics-daemon-hjstr\" (UID: \"d7a0204c-3f9f-4f10-ace5-def5e62925dd\") " pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:04.831609 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:04.831503    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 23:11:04.831609 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:04.831550    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 23:11:04.831609 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:04.831587    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret podName:e1d3f804-fd8f-43fa-81ab-82a96cc020e0 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:12.831565471 +0000 UTC m=+18.085355460 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret") pod "global-pull-secret-syncer-vdlx9" (UID: "e1d3f804-fd8f-43fa-81ab-82a96cc020e0") : object "kube-system"/"original-pull-secret" not registered
Jul 02 23:11:04.831609 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:04.831608    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs podName:d7a0204c-3f9f-4f10-ace5-def5e62925dd nodeName:}" failed. No retries permitted until 2026-07-02 23:11:12.831598277 +0000 UTC m=+18.085388284 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs") pod "network-metrics-daemon-hjstr" (UID: "d7a0204c-3f9f-4f10-ace5-def5e62925dd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 23:11:05.033197 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:05.033152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5kd\" (UniqueName: \"kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd\") pod \"network-check-target-l9lgl\" (UID: \"8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd\") " pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:05.033384 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:05.033337    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 23:11:05.033384 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:05.033364    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 23:11:05.033384 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:05.033377    2580 projected.go:196] Error preparing data for projected volume kube-api-access-cn5kd for pod openshift-network-diagnostics/network-check-target-l9lgl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 23:11:05.033557 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:05.033443    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd podName:8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd nodeName:}" failed. No retries permitted until 2026-07-02 23:11:13.033424741 +0000 UTC m=+18.287214722 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cn5kd" (UniqueName: "kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd") pod "network-check-target-l9lgl" (UID: "8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 23:11:05.279179 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:05.278512    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:05.279179 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:05.278644    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:05.279179 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:05.278697    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:05.279179 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:05.278763    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:06.277765 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:06.277143    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:06.277765 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:06.277298    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:07.278095 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:07.278059    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:07.278095 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:07.278090    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:07.278571 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:07.278241    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:07.278571 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:07.278362    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:08.277039 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:08.277008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:08.277203 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:08.277139    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:09.277387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:09.277341    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:09.277862 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:09.277395    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:09.277862 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:09.277493    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:09.277862 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:09.277603    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:10.277866 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:10.277824    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:10.278314 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:10.277947    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:11.277881 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:11.277844    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:11.278360 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:11.277861    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:11.278360 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:11.277987    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:11.278360 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:11.278045    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:12.277364 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:12.277324    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:12.277574 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:12.277469    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:12.894245 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:12.894152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:12.894245 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:12.894211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs\") pod \"network-metrics-daemon-hjstr\" (UID: \"d7a0204c-3f9f-4f10-ace5-def5e62925dd\") " pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:12.894741 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:12.894310    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 23:11:12.894741 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:12.894371    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs podName:d7a0204c-3f9f-4f10-ace5-def5e62925dd nodeName:}" failed. No retries permitted until 2026-07-02 23:11:28.894353038 +0000 UTC m=+34.148143029 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs") pod "network-metrics-daemon-hjstr" (UID: "d7a0204c-3f9f-4f10-ace5-def5e62925dd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 23:11:12.894741 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:12.894310    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 23:11:12.894741 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:12.894471    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret podName:e1d3f804-fd8f-43fa-81ab-82a96cc020e0 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:28.894447265 +0000 UTC m=+34.148237243 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret") pod "global-pull-secret-syncer-vdlx9" (UID: "e1d3f804-fd8f-43fa-81ab-82a96cc020e0") : object "kube-system"/"original-pull-secret" not registered
Jul 02 23:11:13.095617 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:13.095578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5kd\" (UniqueName: \"kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd\") pod \"network-check-target-l9lgl\" (UID: \"8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd\") " pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:13.095814 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:13.095792    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 23:11:13.095874 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:13.095823    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 23:11:13.095874 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:13.095838    2580 projected.go:196] Error preparing data for projected volume kube-api-access-cn5kd for pod openshift-network-diagnostics/network-check-target-l9lgl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 23:11:13.095974 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:13.095905    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd podName:8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd nodeName:}" failed. No retries permitted until 2026-07-02 23:11:29.095885443 +0000 UTC m=+34.349675440 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cn5kd" (UniqueName: "kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd") pod "network-check-target-l9lgl" (UID: "8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 23:11:13.277068 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:13.276977    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:13.277068 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:13.277032    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:13.277275 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:13.277155    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:13.277275 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:13.277257    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:14.277396 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:14.277351    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:14.277932 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:14.277499    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:15.278655 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:15.278604    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:15.279102 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:15.278721    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:15.279102 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:15.278776    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:15.279102 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:15.278862    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:16.277663 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:16.277624    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:16.277815 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:16.277724    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:17.278158 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.277833    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:17.278842 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.277833    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:17.278842 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:17.278282    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:17.278842 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:17.278333    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:17.371506 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.371470    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal" event={"ID":"00f9bd621d84080c84f579c3e648158f","Type":"ContainerStarted","Data":"fe2122a4046a12033b67cc1fa4921abad88c6d71258ef6a5ea3ac2bac64bd7e2"}
Jul 02 23:11:17.372955 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.372928    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-54qq8" event={"ID":"b9e5b05d-952a-4a24-8221-9c64bfa92ec8","Type":"ContainerStarted","Data":"9268128f01f413557a51b6ced84de2894738a26073628fb8ca6fdee65bb54ce7"}
Jul 02 23:11:17.374378 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.374349    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj" event={"ID":"cd52fd45-f3e3-47fb-9725-c1fd501e104b","Type":"ContainerStarted","Data":"82a034e3fa329d3a8b8efdd3742c3ea98be7cd9e08de5d5088d2cc1f0d1ef197"}
Jul 02 23:11:17.378058 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.378029    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc" event={"ID":"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f","Type":"ContainerStarted","Data":"0eff5f8f73a9745277cf14f5041fd9f6cf73f65819efe1446f793596cf232304"}
Jul 02 23:11:17.378192 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.378144    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc" event={"ID":"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f","Type":"ContainerStarted","Data":"ee62f211a7fed33da6deb5a6244f1ea1bff2b598f6d351456e175a5fb961c2d4"}
Jul 02 23:11:17.378259 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.378196    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc" event={"ID":"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f","Type":"ContainerStarted","Data":"732f346a21d2fb6d0b0612629693f08dbdb61e2549d2c8a49025ef4225479c51"}
Jul 02 23:11:17.378259 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.378247    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc" event={"ID":"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f","Type":"ContainerStarted","Data":"2d047c1f9d6654f169dbf7c2f8ee6c44f892cf6c8b24297c01817cb895954bed"}
Jul 02 23:11:17.378330 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.378296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc" event={"ID":"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f","Type":"ContainerStarted","Data":"91a5647c6b089a5a3c69cf0df8573286e21fd2cfa963f7b7e5c4ad817f94d7e5"}
Jul 02 23:11:17.378372 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.378344    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc" event={"ID":"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f","Type":"ContainerStarted","Data":"348ff65bbd3e612a3bdb3d1a0472f14d259415780f3908b5e1bdf9b5b527444a"}
Jul 02 23:11:17.379897 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.379868    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dvdgd" event={"ID":"c0c3ed8f-cc22-4981-8a71-fda36e2b9cf7","Type":"ContainerStarted","Data":"bc3780ab040db6e608ae09fe4e9cd72fc9dd863b75c3f18b69d0e3ec17967f0f"}
Jul 02 23:11:17.381407 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.381381    2580 generic.go:356] "Generic (PLEG): container finished" podID="eed7310e-b623-4eca-91a1-c4c6bc3df056" containerID="2a9fe1d2cca59d9f0d301b6cdf1dfef5489406a13b57a889f63ba87a5563efa1" exitCode=0
Jul 02 23:11:17.381503 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.381460    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jsxq" event={"ID":"eed7310e-b623-4eca-91a1-c4c6bc3df056","Type":"ContainerDied","Data":"2a9fe1d2cca59d9f0d301b6cdf1dfef5489406a13b57a889f63ba87a5563efa1"}
Jul 02 23:11:17.383087 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.383019    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vhqb2" event={"ID":"72a6430a-6d3b-489d-afb8-465243b1bf36","Type":"ContainerStarted","Data":"5062908793b2ab7ae889637e0780483ec2d324688966762c563375fbec80b54d"}
Jul 02 23:11:17.384515 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.384491    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m8pqn" event={"ID":"209988dd-4347-4ac0-ba98-2afa7b5ce94b","Type":"ContainerStarted","Data":"b1c51bf48f25db87a57d818c63c9f1d2dc0674288a5a3d01106713637e58e61f"}
Jul 02 23:11:17.385793 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.385760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-thnfg" event={"ID":"220907ed-e9d9-4b0c-9b89-7be9fdd4552e","Type":"ContainerStarted","Data":"7458211f3c25349c3216dcc69f95e1fec1d61386f27f3650cfc3e693ab4a997a"}
Jul 02 23:11:17.406220 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.406158    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-52.ec2.internal" podStartSLOduration=21.406137125 podStartE2EDuration="21.406137125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:17.389357458 +0000 UTC m=+22.643147457" watchObservedRunningTime="2026-07-02 23:11:17.406137125 +0000 UTC m=+22.659927117"
Jul 02 23:11:17.428259 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.428202    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-thnfg" podStartSLOduration=3.612901466 podStartE2EDuration="22.428185261s" totalImagesPullingTime="18.815283795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:17.405470105 +0000 UTC m=+22.659260105" watchObservedRunningTime="2026-07-02 23:11:17.428185261 +0000 UTC m=+22.681975259"
Jul 02 23:11:17.428485 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.428448    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dvdgd" podStartSLOduration=2.015484931 podStartE2EDuration="21.428437319s" totalImagesPullingTime="19.412952388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:17.428307876 +0000 UTC m=+22.682097865" watchObservedRunningTime="2026-07-02 23:11:17.428437319 +0000 UTC m=+22.682227317"
Jul 02 23:11:17.447292 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.447219    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m8pqn" podStartSLOduration=2.338915217 podStartE2EDuration="21.447202598s" totalImagesPullingTime="19.108287381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:17.447088188 +0000 UTC m=+22.700878184" watchObservedRunningTime="2026-07-02 23:11:17.447202598 +0000 UTC m=+22.700992597"
Jul 02 23:11:17.492095 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.492029    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vhqb2" podStartSLOduration=2.424911382 podStartE2EDuration="21.492006522s" totalImagesPullingTime="19.06709514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:17.491313651 +0000 UTC m=+22.745103673" watchObservedRunningTime="2026-07-02 23:11:17.492006522 +0000 UTC m=+22.745796520"
Jul 02 23:11:17.510624 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:17.510563    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-54qq8" podStartSLOduration=6.457702884 podStartE2EDuration="21.510539994s" totalImagesPullingTime="15.05283711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:17.509606248 +0000 UTC m=+22.763396247" watchObservedRunningTime="2026-07-02 23:11:17.510539994 +0000 UTC m=+22.764329997"
Jul 02 23:11:18.208510 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:18.208461    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 02 23:11:18.277329 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:18.277297    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:18.277511 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:18.277408    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:18.390655 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:18.390558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g5qvg" event={"ID":"5b81881b-bb30-4123-bfdc-e23c11f55ba1","Type":"ContainerStarted","Data":"cddb38790efc186e01ed38ab27f9f2433d370f33b8e301414e3a77fc05d8a2fe"}
Jul 02 23:11:18.392705 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:18.392674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj" event={"ID":"cd52fd45-f3e3-47fb-9725-c1fd501e104b","Type":"ContainerStarted","Data":"ecf5e2cfd10074d07c838c9254b961de27f114e41d5ef286cf7a4bc68bea6616"}
Jul 02 23:11:18.413897 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:18.413843    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-g5qvg" podStartSLOduration=3.274721912 podStartE2EDuration="22.413827449s" totalImagesPullingTime="19.139105537s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:18.413032148 +0000 UTC m=+23.666822175" watchObservedRunningTime="2026-07-02 23:11:18.413827449 +0000 UTC m=+23.667617447"
Jul 02 23:11:18.930774 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:18.930599    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T23:11:18.208485445Z","UUID":"d39aef88-7812-492e-bace-e5e44d8507d8","Handler":null,"Name":"","Endpoint":""}
Jul 02 23:11:18.932514 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:18.932473    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 02 23:11:18.932688 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:18.932528    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 02 23:11:19.276993 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:19.276900    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:19.276993 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:19.276910    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:19.277205 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:19.277049    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:19.277205 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:19.277093    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:20.261527 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:20.261337    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-thnfg"
Jul 02 23:11:20.262036 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:20.262016    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-thnfg"
Jul 02 23:11:20.276962 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:20.276925    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:20.277147 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:20.277042    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:20.400256 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:20.400218    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj" event={"ID":"cd52fd45-f3e3-47fb-9725-c1fd501e104b","Type":"ContainerStarted","Data":"df1778541b079ad549b024ca99cd27616256a9141c05a0e28d4ac4a7e3a72f54"}
Jul 02 23:11:20.404198 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:20.404090    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc" event={"ID":"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f","Type":"ContainerStarted","Data":"1f29c026aba485091d3db803cc28e83705a0285f617869c97ca3fcdac37f8965"}
Jul 02 23:11:20.404198 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:20.404200    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-thnfg"
Jul 02 23:11:20.404814 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:20.404795    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-thnfg"
Jul 02 23:11:20.420260 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:20.420210    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmbnj" podStartSLOduration=3.051864606 podStartE2EDuration="24.420193937s" totalImagesPullingTime="21.368329331s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:20.41989925 +0000 UTC m=+25.673689250" watchObservedRunningTime="2026-07-02 23:11:20.420193937 +0000 UTC m=+25.673983936"
Jul 02 23:11:21.281072 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:21.281040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:21.281596 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:21.281040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:21.281596 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:21.281216    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:21.281596 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:21.281325    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:22.278010 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:22.277757    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:22.278188 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:22.278015    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:22.411980 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:22.411941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc" event={"ID":"da9201f5-a1c4-4e71-b7f8-beb4ef769b4f","Type":"ContainerStarted","Data":"0ff10c4d25bebddba1b7bcabf9e3b02661071bc6259f5c88f2a6519cb5a5d77e"}
Jul 02 23:11:22.412865 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:22.412193    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:11:22.413852 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:22.413822    2580 generic.go:356] "Generic (PLEG): container finished" podID="eed7310e-b623-4eca-91a1-c4c6bc3df056" containerID="3fcf60b54e292c232c140fd75fe30aaea17057cd25f488ef8177988926f45a12" exitCode=0
Jul 02 23:11:22.413973 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:22.413897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jsxq" event={"ID":"eed7310e-b623-4eca-91a1-c4c6bc3df056","Type":"ContainerDied","Data":"3fcf60b54e292c232c140fd75fe30aaea17057cd25f488ef8177988926f45a12"}
Jul 02 23:11:22.428544 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:22.428520    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:11:22.446615 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:22.446563    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc" podStartSLOduration=7.301114818 podStartE2EDuration="26.446547913s" totalImagesPullingTime="19.145433095s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:22.446039685 +0000 UTC m=+27.699829683" watchObservedRunningTime="2026-07-02 23:11:22.446547913 +0000 UTC m=+27.700337911"
Jul 02 23:11:23.277356 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:23.277326    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:23.277356 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:23.277365    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:23.277675 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:23.277453    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:23.277675 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:23.277574    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:23.417047 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:23.417004    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:11:23.417455 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:23.417104    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:11:23.431942 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:23.431914    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:11:23.630715 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:23.630670    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vdlx9"]
Jul 02 23:11:23.630890 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:23.630811    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:23.630934 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:23.630913    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:23.633496 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:23.633467    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l9lgl"]
Jul 02 23:11:23.633638 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:23.633592    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:23.633789 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:23.633719    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:23.634425 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:23.634166    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hjstr"]
Jul 02 23:11:23.634425 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:23.634277    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:23.634425 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:23.634378    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:24.420666 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:24.420633    2580 generic.go:356] "Generic (PLEG): container finished" podID="eed7310e-b623-4eca-91a1-c4c6bc3df056" containerID="0f300247400c7ebc3c7dbcc1afa3b4759b4e42e3f10a16887ae9142ea4965047" exitCode=0
Jul 02 23:11:24.421157 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:24.420726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jsxq" event={"ID":"eed7310e-b623-4eca-91a1-c4c6bc3df056","Type":"ContainerDied","Data":"0f300247400c7ebc3c7dbcc1afa3b4759b4e42e3f10a16887ae9142ea4965047"}
Jul 02 23:11:25.279334 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:25.278694    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:25.279334 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:25.279043    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:25.279591 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:25.279548    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:25.281022 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:25.279683    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:25.281022 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:25.279756    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:25.281022 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:25.279889    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:26.427193 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:26.427159    2580 generic.go:356] "Generic (PLEG): container finished" podID="eed7310e-b623-4eca-91a1-c4c6bc3df056" containerID="e9415bd318831760deddaec03e32471f0411454166fabb3a65a30b579490adba" exitCode=0
Jul 02 23:11:26.427680 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:26.427211    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jsxq" event={"ID":"eed7310e-b623-4eca-91a1-c4c6bc3df056","Type":"ContainerDied","Data":"e9415bd318831760deddaec03e32471f0411454166fabb3a65a30b579490adba"}
Jul 02 23:11:27.277721 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:27.277679    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:27.277919 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:27.277681    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:27.277919 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:27.277845    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:27.278032 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:27.277679    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:27.278085 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:27.277957    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:27.278085 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:27.278046    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:28.921178 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:28.921134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:28.921178 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:28.921187    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs\") pod \"network-metrics-daemon-hjstr\" (UID: \"d7a0204c-3f9f-4f10-ace5-def5e62925dd\") " pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:28.921841 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:28.921292    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 02 23:11:28.921841 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:28.921309    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 23:11:28.921841 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:28.921368    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs podName:d7a0204c-3f9f-4f10-ace5-def5e62925dd nodeName:}" failed. No retries permitted until 2026-07-02 23:12:00.921352817 +0000 UTC m=+66.175142794 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs") pod "network-metrics-daemon-hjstr" (UID: "d7a0204c-3f9f-4f10-ace5-def5e62925dd") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 02 23:11:28.921841 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:28.921387    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret podName:e1d3f804-fd8f-43fa-81ab-82a96cc020e0 nodeName:}" failed. No retries permitted until 2026-07-02 23:12:00.921377684 +0000 UTC m=+66.175167661 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret") pod "global-pull-secret-syncer-vdlx9" (UID: "e1d3f804-fd8f-43fa-81ab-82a96cc020e0") : object "kube-system"/"original-pull-secret" not registered
Jul 02 23:11:29.122823 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.122779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5kd\" (UniqueName: \"kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd\") pod \"network-check-target-l9lgl\" (UID: \"8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd\") " pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:29.123009 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:29.122979    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 02 23:11:29.123009 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:29.123005    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 02 23:11:29.123145 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:29.123018    2580 projected.go:196] Error preparing data for projected volume kube-api-access-cn5kd for pod openshift-network-diagnostics/network-check-target-l9lgl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 23:11:29.123145 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:29.123085    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd podName:8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd nodeName:}" failed. No retries permitted until 2026-07-02 23:12:01.123065327 +0000 UTC m=+66.376855352 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cn5kd" (UniqueName: "kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd") pod "network-check-target-l9lgl" (UID: "8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 02 23:11:29.277861 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.277769    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:29.278025 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.277770    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:29.278025 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:29.277919    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vdlx9" podUID="e1d3f804-fd8f-43fa-81ab-82a96cc020e0"
Jul 02 23:11:29.278025 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.277777    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:29.278207 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:29.278016    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjstr" podUID="d7a0204c-3f9f-4f10-ace5-def5e62925dd"
Jul 02 23:11:29.278207 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:29.278069    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9lgl" podUID="8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd"
Jul 02 23:11:29.498558 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.498532    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-52.ec2.internal" event="NodeReady"
Jul 02 23:11:29.498733 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.498691    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 02 23:11:29.564827 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.564752    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"]
Jul 02 23:11:29.567133 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.567095    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.570916 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.570890    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 02 23:11:29.571069 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.570992    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kcbnf\""
Jul 02 23:11:29.571069 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.571029    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 02 23:11:29.571309 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.571292    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 02 23:11:29.578881 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.578860    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 02 23:11:29.583114 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.583091    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lhb24"]
Jul 02 23:11:29.585237 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.585206    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:29.585977 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.585955    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"]
Jul 02 23:11:29.588443 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.588419    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 02 23:11:29.589705 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.588742    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2clb5\""
Jul 02 23:11:29.589705 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.589552    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 02 23:11:29.591922 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.591439    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5j8vq"]
Jul 02 23:11:29.594044 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.594019    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5j8vq"
Jul 02 23:11:29.597135 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.597102    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 02 23:11:29.597247 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.597136    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 02 23:11:29.597247 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.597112    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rpvf9\""
Jul 02 23:11:29.597247 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.597112    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 02 23:11:29.602839 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.602808    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lhb24"]
Jul 02 23:11:29.606467 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.606445    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5j8vq"]
Jul 02 23:11:29.726853 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.726816    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d07d35c0-4de5-48b9-b2c2-890e8f479e17-tmp-dir\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:29.726853 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.726864    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.727109 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.726899    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/353a13fa-fc42-4342-a702-35ac283cd9b2-installation-pull-secrets\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.727109 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.726923    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-bound-sa-token\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.727109 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.726947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frjd5\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-kube-api-access-frjd5\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.727109 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.727069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/353a13fa-fc42-4342-a702-35ac283cd9b2-trusted-ca\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.727338 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.727101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/353a13fa-fc42-4342-a702-35ac283cd9b2-image-registry-private-configuration\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.727338 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.727154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-certificates\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.727338 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.727178    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klxvk\" (UniqueName: \"kubernetes.io/projected/10eb02f1-db27-4000-87ee-07331d835fda-kube-api-access-klxvk\") pod \"ingress-canary-5j8vq\" (UID: \"10eb02f1-db27-4000-87ee-07331d835fda\") " pod="openshift-ingress-canary/ingress-canary-5j8vq"
Jul 02 23:11:29.727338 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.727208    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d07d35c0-4de5-48b9-b2c2-890e8f479e17-config-volume\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:29.727338 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.727307    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:29.727564 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.727364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cff5w\" (UniqueName: \"kubernetes.io/projected/d07d35c0-4de5-48b9-b2c2-890e8f479e17-kube-api-access-cff5w\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:29.727564 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.727406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/353a13fa-fc42-4342-a702-35ac283cd9b2-ca-trust-extracted\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.727564 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.727442    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10eb02f1-db27-4000-87ee-07331d835fda-cert\") pod \"ingress-canary-5j8vq\" (UID: \"10eb02f1-db27-4000-87ee-07331d835fda\") " pod="openshift-ingress-canary/ingress-canary-5j8vq"
Jul 02 23:11:29.828264 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.828170    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10eb02f1-db27-4000-87ee-07331d835fda-cert\") pod \"ingress-canary-5j8vq\" (UID: \"10eb02f1-db27-4000-87ee-07331d835fda\") " pod="openshift-ingress-canary/ingress-canary-5j8vq"
Jul 02 23:11:29.828264 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.828229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d07d35c0-4de5-48b9-b2c2-890e8f479e17-tmp-dir\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:29.828476 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.828363    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.828476 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.828418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/353a13fa-fc42-4342-a702-35ac283cd9b2-installation-pull-secrets\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.828476 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.828447    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-bound-sa-token\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.828476 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.828472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frjd5\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-kube-api-access-frjd5\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.828644 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:29.828492    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 23:11:29.828644 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:29.828510    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7df9ff9cd7-4fn8r: secret "image-registry-tls" not found
Jul 02 23:11:29.828644 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.828517    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/353a13fa-fc42-4342-a702-35ac283cd9b2-trusted-ca\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.828644 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.828559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/353a13fa-fc42-4342-a702-35ac283cd9b2-image-registry-private-configuration\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.828644 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.828567    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d07d35c0-4de5-48b9-b2c2-890e8f479e17-tmp-dir\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:29.828644 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:29.828594    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls podName:353a13fa-fc42-4342-a702-35ac283cd9b2 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:30.328572547 +0000 UTC m=+35.582362537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls") pod "image-registry-7df9ff9cd7-4fn8r" (UID: "353a13fa-fc42-4342-a702-35ac283cd9b2") : secret "image-registry-tls" not found
Jul 02 23:11:29.828644 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.828618    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-certificates\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.828932 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.828660    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klxvk\" (UniqueName: \"kubernetes.io/projected/10eb02f1-db27-4000-87ee-07331d835fda-kube-api-access-klxvk\") pod \"ingress-canary-5j8vq\" (UID: \"10eb02f1-db27-4000-87ee-07331d835fda\") " pod="openshift-ingress-canary/ingress-canary-5j8vq"
Jul 02 23:11:29.829202 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.829044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d07d35c0-4de5-48b9-b2c2-890e8f479e17-config-volume\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:29.829202 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.829107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:29.829202 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.829189    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cff5w\" (UniqueName: \"kubernetes.io/projected/d07d35c0-4de5-48b9-b2c2-890e8f479e17-kube-api-access-cff5w\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:29.829414 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.829220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/353a13fa-fc42-4342-a702-35ac283cd9b2-ca-trust-extracted\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.829414 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:29.829366    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 23:11:29.829514 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:29.829469    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls podName:d07d35c0-4de5-48b9-b2c2-890e8f479e17 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:30.329450877 +0000 UTC m=+35.583240860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls") pod "dns-default-lhb24" (UID: "d07d35c0-4de5-48b9-b2c2-890e8f479e17") : secret "dns-default-metrics-tls" not found
Jul 02 23:11:29.829578 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.829541    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-certificates\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.829578 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.829565    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d07d35c0-4de5-48b9-b2c2-890e8f479e17-config-volume\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:29.829680 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.829643    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/353a13fa-fc42-4342-a702-35ac283cd9b2-ca-trust-extracted\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.830269 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.830247    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/353a13fa-fc42-4342-a702-35ac283cd9b2-trusted-ca\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.833176 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.833152    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/353a13fa-fc42-4342-a702-35ac283cd9b2-image-registry-private-configuration\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.833176 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.833167    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/353a13fa-fc42-4342-a702-35ac283cd9b2-installation-pull-secrets\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.833334 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.833188    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10eb02f1-db27-4000-87ee-07331d835fda-cert\") pod \"ingress-canary-5j8vq\" (UID: \"10eb02f1-db27-4000-87ee-07331d835fda\") " pod="openshift-ingress-canary/ingress-canary-5j8vq"
Jul 02 23:11:29.840007 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.839808    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-bound-sa-token\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.840339 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.840313    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cff5w\" (UniqueName: \"kubernetes.io/projected/d07d35c0-4de5-48b9-b2c2-890e8f479e17-kube-api-access-cff5w\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:29.840554 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.840523    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klxvk\" (UniqueName: \"kubernetes.io/projected/10eb02f1-db27-4000-87ee-07331d835fda-kube-api-access-klxvk\") pod \"ingress-canary-5j8vq\" (UID: \"10eb02f1-db27-4000-87ee-07331d835fda\") " pod="openshift-ingress-canary/ingress-canary-5j8vq"
Jul 02 23:11:29.842018 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.841997    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frjd5\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-kube-api-access-frjd5\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:29.906882 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:29.906839    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5j8vq"
Jul 02 23:11:30.332770 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:30.332596    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:30.333375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:30.332797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:30.333375 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:30.332746    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 23:11:30.333375 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:30.332882    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls podName:d07d35c0-4de5-48b9-b2c2-890e8f479e17 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:31.332866138 +0000 UTC m=+36.586656116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls") pod "dns-default-lhb24" (UID: "d07d35c0-4de5-48b9-b2c2-890e8f479e17") : secret "dns-default-metrics-tls" not found
Jul 02 23:11:30.333375 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:30.332902    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 23:11:30.333375 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:30.332915    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7df9ff9cd7-4fn8r: secret "image-registry-tls" not found
Jul 02 23:11:30.333375 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:30.332960    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls podName:353a13fa-fc42-4342-a702-35ac283cd9b2 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:31.332948379 +0000 UTC m=+36.586738361 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls") pod "image-registry-7df9ff9cd7-4fn8r" (UID: "353a13fa-fc42-4342-a702-35ac283cd9b2") : secret "image-registry-tls" not found
Jul 02 23:11:30.444852 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:30.444812    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5j8vq"]
Jul 02 23:11:30.448814 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:11:30.448786    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10eb02f1_db27_4000_87ee_07331d835fda.slice/crio-6f986cfe2904db456311c0e4abfa04a919f1a2559e7cb5449c0bd7ef50d77a93 WatchSource:0}: Error finding container 6f986cfe2904db456311c0e4abfa04a919f1a2559e7cb5449c0bd7ef50d77a93: Status 404 returned error can't find the container with id 6f986cfe2904db456311c0e4abfa04a919f1a2559e7cb5449c0bd7ef50d77a93
Jul 02 23:11:31.277707 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:31.277673    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:11:31.277890 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:31.277733    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:11:31.277890 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:31.277740    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:11:31.282146 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:31.282079    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 23:11:31.282146 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:31.282103    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 23:11:31.282146 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:31.282086    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qmzc5\""
Jul 02 23:11:31.282412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:31.282110    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 23:11:31.282412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:31.282079    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 23:11:31.282412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:31.282245    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-872wr\""
Jul 02 23:11:31.341538 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:31.341503    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:31.342002 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:31.341562    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:31.342002 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:31.341685    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 23:11:31.342002 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:31.341700    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 23:11:31.342002 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:31.341719    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7df9ff9cd7-4fn8r: secret "image-registry-tls" not found
Jul 02 23:11:31.342002 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:31.341767    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls podName:d07d35c0-4de5-48b9-b2c2-890e8f479e17 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:33.341746332 +0000 UTC m=+38.595536313 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls") pod "dns-default-lhb24" (UID: "d07d35c0-4de5-48b9-b2c2-890e8f479e17") : secret "dns-default-metrics-tls" not found
Jul 02 23:11:31.342002 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:31.341789    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls podName:353a13fa-fc42-4342-a702-35ac283cd9b2 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:33.341779674 +0000 UTC m=+38.595569656 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls") pod "image-registry-7df9ff9cd7-4fn8r" (UID: "353a13fa-fc42-4342-a702-35ac283cd9b2") : secret "image-registry-tls" not found
Jul 02 23:11:31.442109 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:31.442070    2580 generic.go:356] "Generic (PLEG): container finished" podID="eed7310e-b623-4eca-91a1-c4c6bc3df056" containerID="7fd6beb993b20ecc83924a36eae18860ebc8852babefa8b6d0d4e85c6555f4fe" exitCode=0
Jul 02 23:11:31.442279 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:31.442167    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jsxq" event={"ID":"eed7310e-b623-4eca-91a1-c4c6bc3df056","Type":"ContainerDied","Data":"7fd6beb993b20ecc83924a36eae18860ebc8852babefa8b6d0d4e85c6555f4fe"}
Jul 02 23:11:31.443472 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:31.443438    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5j8vq" event={"ID":"10eb02f1-db27-4000-87ee-07331d835fda","Type":"ContainerStarted","Data":"6f986cfe2904db456311c0e4abfa04a919f1a2559e7cb5449c0bd7ef50d77a93"}
Jul 02 23:11:32.450924 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:32.450885    2580 generic.go:356] "Generic (PLEG): container finished" podID="eed7310e-b623-4eca-91a1-c4c6bc3df056" containerID="2c6ca22f35f9955cb138d61050835b964d35992445716056d912cf70aaf7732e" exitCode=0
Jul 02 23:11:32.451437 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:32.450952    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jsxq" event={"ID":"eed7310e-b623-4eca-91a1-c4c6bc3df056","Type":"ContainerDied","Data":"2c6ca22f35f9955cb138d61050835b964d35992445716056d912cf70aaf7732e"}
Jul 02 23:11:33.359313 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:33.359082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:33.359488 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:33.359354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:33.359488 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:33.359226    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 23:11:33.359488 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:33.359441    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls podName:d07d35c0-4de5-48b9-b2c2-890e8f479e17 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:37.359419063 +0000 UTC m=+42.613209060 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls") pod "dns-default-lhb24" (UID: "d07d35c0-4de5-48b9-b2c2-890e8f479e17") : secret "dns-default-metrics-tls" not found
Jul 02 23:11:33.359488 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:33.359484    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 23:11:33.359631 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:33.359498    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7df9ff9cd7-4fn8r: secret "image-registry-tls" not found
Jul 02 23:11:33.359631 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:33.359533    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls podName:353a13fa-fc42-4342-a702-35ac283cd9b2 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:37.359522096 +0000 UTC m=+42.613312075 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls") pod "image-registry-7df9ff9cd7-4fn8r" (UID: "353a13fa-fc42-4342-a702-35ac283cd9b2") : secret "image-registry-tls" not found
Jul 02 23:11:33.456839 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:33.456742    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jsxq" event={"ID":"eed7310e-b623-4eca-91a1-c4c6bc3df056","Type":"ContainerStarted","Data":"1526c4b51e22391d0f6b12a4d07835a57b03b61c892bec13596b67037de3e3f4"}
Jul 02 23:11:33.458172 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:33.458148    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5j8vq" event={"ID":"10eb02f1-db27-4000-87ee-07331d835fda","Type":"ContainerStarted","Data":"637d44d6b46bdc7ff425fc9bf7e026728bf6508bea5a4211de4359e791e26af3"}
Jul 02 23:11:33.485709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:33.485654    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5jsxq" podStartSLOduration=8.023769036 podStartE2EDuration="37.485634918s" totalImagesPullingTime="29.461865882s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:33.485163673 +0000 UTC m=+38.738953673" watchObservedRunningTime="2026-07-02 23:11:33.485634918 +0000 UTC m=+38.739424919"
Jul 02 23:11:33.503310 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:33.503183    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5j8vq" podStartSLOduration=2.11505672 podStartE2EDuration="4.503162581s" totalImagesPullingTime="2.388105861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:33.50308657 +0000 UTC m=+38.756876571" watchObservedRunningTime="2026-07-02 23:11:33.503162581 +0000 UTC m=+38.756952581"
Jul 02 23:11:37.388464 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:37.388411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:37.388926 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:37.388492    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:37.388926 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:37.388571    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 23:11:37.388926 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:37.388585    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 23:11:37.388926 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:37.388591    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7df9ff9cd7-4fn8r: secret "image-registry-tls" not found
Jul 02 23:11:37.388926 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:37.388638    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls podName:d07d35c0-4de5-48b9-b2c2-890e8f479e17 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:45.388624229 +0000 UTC m=+50.642414211 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls") pod "dns-default-lhb24" (UID: "d07d35c0-4de5-48b9-b2c2-890e8f479e17") : secret "dns-default-metrics-tls" not found
Jul 02 23:11:37.388926 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:37.388652    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls podName:353a13fa-fc42-4342-a702-35ac283cd9b2 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:45.388645723 +0000 UTC m=+50.642435699 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls") pod "image-registry-7df9ff9cd7-4fn8r" (UID: "353a13fa-fc42-4342-a702-35ac283cd9b2") : secret "image-registry-tls" not found
Jul 02 23:11:38.480602 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.480574    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-f7h7z"]
Jul 02 23:11:38.511967 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.511928    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f7h7z"]
Jul 02 23:11:38.512114 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.512061    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:38.514750 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.514718    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6w77h\""
Jul 02 23:11:38.516067 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.516023    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 02 23:11:38.516067 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.516035    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 02 23:11:38.516067 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.516035    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 02 23:11:38.516067 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.516057    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 02 23:11:38.598417 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.598377    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/98bf3fef-ce15-4556-abac-89874cf50e09-crio-socket\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:38.598417 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.598426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvqqw\" (UniqueName: \"kubernetes.io/projected/98bf3fef-ce15-4556-abac-89874cf50e09-kube-api-access-xvqqw\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:38.598618 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.598482    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/98bf3fef-ce15-4556-abac-89874cf50e09-data-volume\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:38.598618 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.598520    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98bf3fef-ce15-4556-abac-89874cf50e09-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:38.598618 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.598536    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:38.698843 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.698809    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/98bf3fef-ce15-4556-abac-89874cf50e09-data-volume\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:38.698843 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.698845    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98bf3fef-ce15-4556-abac-89874cf50e09-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:38.699106 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.698863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:38.699106 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.698915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/98bf3fef-ce15-4556-abac-89874cf50e09-crio-socket\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:38.699106 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.698935    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvqqw\" (UniqueName: \"kubernetes.io/projected/98bf3fef-ce15-4556-abac-89874cf50e09-kube-api-access-xvqqw\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:38.699106 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:38.698982    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 23:11:38.699106 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:38.699011    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 23:11:38.699106 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:38.699073    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98bf3fef-ce15-4556-abac-89874cf50e09-kube-rbac-proxy-cm podName:98bf3fef-ce15-4556-abac-89874cf50e09 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:39.199050634 +0000 UTC m=+44.452840643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/98bf3fef-ce15-4556-abac-89874cf50e09-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f7h7z" (UID: "98bf3fef-ce15-4556-abac-89874cf50e09") : configmap "kube-rbac-proxy" not found
Jul 02 23:11:38.699106 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.699087    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/98bf3fef-ce15-4556-abac-89874cf50e09-crio-socket\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:38.699106 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:38.699097    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls podName:98bf3fef-ce15-4556-abac-89874cf50e09 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:39.199087476 +0000 UTC m=+44.452877457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f7h7z" (UID: "98bf3fef-ce15-4556-abac-89874cf50e09") : secret "insights-runtime-extractor-tls" not found
Jul 02 23:11:38.699428 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.699256    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/98bf3fef-ce15-4556-abac-89874cf50e09-data-volume\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:38.719368 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:38.719338    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvqqw\" (UniqueName: \"kubernetes.io/projected/98bf3fef-ce15-4556-abac-89874cf50e09-kube-api-access-xvqqw\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:39.149538 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:39.149503    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-5jwtj"]
Jul 02 23:11:39.164497 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:39.164454    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-5jwtj"]
Jul 02 23:11:39.164663 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:39.164602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5jwtj"
Jul 02 23:11:39.167806 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:39.167781    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 02 23:11:39.169237 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:39.169214    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 02 23:11:39.169383 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:39.169214    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2mn4j\""
Jul 02 23:11:39.202203 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:39.202160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98bf3fef-ce15-4556-abac-89874cf50e09-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:39.202203 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:39.202199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:39.202405 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:39.202287    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 23:11:39.202405 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:39.202360    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 23:11:39.202405 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:39.202364    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98bf3fef-ce15-4556-abac-89874cf50e09-kube-rbac-proxy-cm podName:98bf3fef-ce15-4556-abac-89874cf50e09 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:40.20234666 +0000 UTC m=+45.456136641 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/98bf3fef-ce15-4556-abac-89874cf50e09-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f7h7z" (UID: "98bf3fef-ce15-4556-abac-89874cf50e09") : configmap "kube-rbac-proxy" not found
Jul 02 23:11:39.202504 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:39.202428    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls podName:98bf3fef-ce15-4556-abac-89874cf50e09 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:40.202397651 +0000 UTC m=+45.456187628 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f7h7z" (UID: "98bf3fef-ce15-4556-abac-89874cf50e09") : secret "insights-runtime-extractor-tls" not found
Jul 02 23:11:39.303265 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:39.303224    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpjzm\" (UniqueName: \"kubernetes.io/projected/e4e0cee9-b9cf-4a41-9e60-6592f430b731-kube-api-access-cpjzm\") pod \"migrator-d57d58cc9-5jwtj\" (UID: \"e4e0cee9-b9cf-4a41-9e60-6592f430b731\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5jwtj"
Jul 02 23:11:39.403742 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:39.403664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpjzm\" (UniqueName: \"kubernetes.io/projected/e4e0cee9-b9cf-4a41-9e60-6592f430b731-kube-api-access-cpjzm\") pod \"migrator-d57d58cc9-5jwtj\" (UID: \"e4e0cee9-b9cf-4a41-9e60-6592f430b731\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5jwtj"
Jul 02 23:11:39.412921 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:39.412892    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpjzm\" (UniqueName: \"kubernetes.io/projected/e4e0cee9-b9cf-4a41-9e60-6592f430b731-kube-api-access-cpjzm\") pod \"migrator-d57d58cc9-5jwtj\" (UID: \"e4e0cee9-b9cf-4a41-9e60-6592f430b731\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5jwtj"
Jul 02 23:11:39.474204 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:39.474171    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5jwtj"
Jul 02 23:11:39.613866 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:39.613824    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-5jwtj"]
Jul 02 23:11:39.617466 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:11:39.617438    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4e0cee9_b9cf_4a41_9e60_6592f430b731.slice/crio-8a07ecc91262c0f6328ad219820f9e99691297986261aab15dbce527c6d4f0d9 WatchSource:0}: Error finding container 8a07ecc91262c0f6328ad219820f9e99691297986261aab15dbce527c6d4f0d9: Status 404 returned error can't find the container with id 8a07ecc91262c0f6328ad219820f9e99691297986261aab15dbce527c6d4f0d9
Jul 02 23:11:40.211070 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.211027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98bf3fef-ce15-4556-abac-89874cf50e09-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:40.211070 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.211082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:40.211350 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:40.211176    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 23:11:40.211350 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:40.211228    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 23:11:40.211350 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:40.211268    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98bf3fef-ce15-4556-abac-89874cf50e09-kube-rbac-proxy-cm podName:98bf3fef-ce15-4556-abac-89874cf50e09 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:42.211242541 +0000 UTC m=+47.465032531 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/98bf3fef-ce15-4556-abac-89874cf50e09-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f7h7z" (UID: "98bf3fef-ce15-4556-abac-89874cf50e09") : configmap "kube-rbac-proxy" not found
Jul 02 23:11:40.211350 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:40.211287    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls podName:98bf3fef-ce15-4556-abac-89874cf50e09 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:42.211278622 +0000 UTC m=+47.465068599 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f7h7z" (UID: "98bf3fef-ce15-4556-abac-89874cf50e09") : secret "insights-runtime-extractor-tls" not found
Jul 02 23:11:40.275169 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.275134    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vhqb2_72a6430a-6d3b-489d-afb8-465243b1bf36/dns-node-resolver/0.log"
Jul 02 23:11:40.476499 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.476405    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5jwtj" event={"ID":"e4e0cee9-b9cf-4a41-9e60-6592f430b731","Type":"ContainerStarted","Data":"8a07ecc91262c0f6328ad219820f9e99691297986261aab15dbce527c6d4f0d9"}
Jul 02 23:11:40.597625 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.597587    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-gdh7c"]
Jul 02 23:11:40.620486 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.620456    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-gdh7c"]
Jul 02 23:11:40.620937 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.620559    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-gdh7c"
Jul 02 23:11:40.623434 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.623403    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 02 23:11:40.623434 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.623433    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 02 23:11:40.623602 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.623493    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 02 23:11:40.624750 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.624723    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-kjnzn\""
Jul 02 23:11:40.624872 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.624830    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 02 23:11:40.715336 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.715304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6aa74e11-1192-418e-86fc-9a64d41ed398-signing-key\") pod \"service-ca-758f599c5-gdh7c\" (UID: \"6aa74e11-1192-418e-86fc-9a64d41ed398\") " pod="openshift-service-ca/service-ca-758f599c5-gdh7c"
Jul 02 23:11:40.715546 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.715440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6aa74e11-1192-418e-86fc-9a64d41ed398-signing-cabundle\") pod \"service-ca-758f599c5-gdh7c\" (UID: \"6aa74e11-1192-418e-86fc-9a64d41ed398\") " pod="openshift-service-ca/service-ca-758f599c5-gdh7c"
Jul 02 23:11:40.715546 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.715510    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw9hr\" (UniqueName: \"kubernetes.io/projected/6aa74e11-1192-418e-86fc-9a64d41ed398-kube-api-access-pw9hr\") pod \"service-ca-758f599c5-gdh7c\" (UID: \"6aa74e11-1192-418e-86fc-9a64d41ed398\") " pod="openshift-service-ca/service-ca-758f599c5-gdh7c"
Jul 02 23:11:40.816758 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.816652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6aa74e11-1192-418e-86fc-9a64d41ed398-signing-cabundle\") pod \"service-ca-758f599c5-gdh7c\" (UID: \"6aa74e11-1192-418e-86fc-9a64d41ed398\") " pod="openshift-service-ca/service-ca-758f599c5-gdh7c"
Jul 02 23:11:40.816758 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.816734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw9hr\" (UniqueName: \"kubernetes.io/projected/6aa74e11-1192-418e-86fc-9a64d41ed398-kube-api-access-pw9hr\") pod \"service-ca-758f599c5-gdh7c\" (UID: \"6aa74e11-1192-418e-86fc-9a64d41ed398\") " pod="openshift-service-ca/service-ca-758f599c5-gdh7c"
Jul 02 23:11:40.816965 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.816770    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6aa74e11-1192-418e-86fc-9a64d41ed398-signing-key\") pod \"service-ca-758f599c5-gdh7c\" (UID: \"6aa74e11-1192-418e-86fc-9a64d41ed398\") " pod="openshift-service-ca/service-ca-758f599c5-gdh7c"
Jul 02 23:11:40.817566 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.817534    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6aa74e11-1192-418e-86fc-9a64d41ed398-signing-cabundle\") pod \"service-ca-758f599c5-gdh7c\" (UID: \"6aa74e11-1192-418e-86fc-9a64d41ed398\") " pod="openshift-service-ca/service-ca-758f599c5-gdh7c"
Jul 02 23:11:40.819387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.819365    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6aa74e11-1192-418e-86fc-9a64d41ed398-signing-key\") pod \"service-ca-758f599c5-gdh7c\" (UID: \"6aa74e11-1192-418e-86fc-9a64d41ed398\") " pod="openshift-service-ca/service-ca-758f599c5-gdh7c"
Jul 02 23:11:40.828008 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.827982    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw9hr\" (UniqueName: \"kubernetes.io/projected/6aa74e11-1192-418e-86fc-9a64d41ed398-kube-api-access-pw9hr\") pod \"service-ca-758f599c5-gdh7c\" (UID: \"6aa74e11-1192-418e-86fc-9a64d41ed398\") " pod="openshift-service-ca/service-ca-758f599c5-gdh7c"
Jul 02 23:11:40.875187 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.875154    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-54qq8_b9e5b05d-952a-4a24-8221-9c64bfa92ec8/node-ca/0.log"
Jul 02 23:11:40.931181 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:40.931145    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-gdh7c"
Jul 02 23:11:41.155269 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:41.155239    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-gdh7c"]
Jul 02 23:11:41.158823 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:11:41.158793    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aa74e11_1192_418e_86fc_9a64d41ed398.slice/crio-3553210d8d2d614d6b3976049cec9f6724295b87193b79df44215493cf818295 WatchSource:0}: Error finding container 3553210d8d2d614d6b3976049cec9f6724295b87193b79df44215493cf818295: Status 404 returned error can't find the container with id 3553210d8d2d614d6b3976049cec9f6724295b87193b79df44215493cf818295
Jul 02 23:11:41.479984 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:41.479949    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-gdh7c" event={"ID":"6aa74e11-1192-418e-86fc-9a64d41ed398","Type":"ContainerStarted","Data":"3553210d8d2d614d6b3976049cec9f6724295b87193b79df44215493cf818295"}
Jul 02 23:11:41.674628 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:41.674547    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5j8vq_10eb02f1-db27-4000-87ee-07331d835fda/serve-healthcheck-canary/0.log"
Jul 02 23:11:42.227549 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:42.227513    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98bf3fef-ce15-4556-abac-89874cf50e09-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:42.227549 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:42.227550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:42.227780 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:42.227698    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 02 23:11:42.227780 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:42.227764    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/98bf3fef-ce15-4556-abac-89874cf50e09-kube-rbac-proxy-cm podName:98bf3fef-ce15-4556-abac-89874cf50e09 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:46.227745788 +0000 UTC m=+51.481535785 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/98bf3fef-ce15-4556-abac-89874cf50e09-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f7h7z" (UID: "98bf3fef-ce15-4556-abac-89874cf50e09") : configmap "kube-rbac-proxy" not found
Jul 02 23:11:42.227947 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:42.227795    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 23:11:42.227947 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:42.227842    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls podName:98bf3fef-ce15-4556-abac-89874cf50e09 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:46.227831395 +0000 UTC m=+51.481621392 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f7h7z" (UID: "98bf3fef-ce15-4556-abac-89874cf50e09") : secret "insights-runtime-extractor-tls" not found
Jul 02 23:11:42.484405 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:42.484314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5jwtj" event={"ID":"e4e0cee9-b9cf-4a41-9e60-6592f430b731","Type":"ContainerStarted","Data":"793793e4a34419c89c6b1de810b8e9c2922594cfb8b1316eff28a9bbf9f85e80"}
Jul 02 23:11:42.484405 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:42.484367    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5jwtj" event={"ID":"e4e0cee9-b9cf-4a41-9e60-6592f430b731","Type":"ContainerStarted","Data":"d92cdc2c856db93cc1945e8e409140927c11c0324f216737a33caf9a8b53555f"}
Jul 02 23:11:42.502391 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:42.502331    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-5jwtj" podStartSLOduration=1.735959826 podStartE2EDuration="3.502309251s" totalImagesPullingTime="1.766349425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:42.501466235 +0000 UTC m=+47.755256233" watchObservedRunningTime="2026-07-02 23:11:42.502309251 +0000 UTC m=+47.756099251"
Jul 02 23:11:44.490769 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:44.490728    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-gdh7c" event={"ID":"6aa74e11-1192-418e-86fc-9a64d41ed398","Type":"ContainerStarted","Data":"c0b8e308b0c5bb54b8940d2818dc34d6e4e42a2123531929fdd0d30d6babbe1b"}
Jul 02 23:11:44.520007 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:44.519953    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-gdh7c" podStartSLOduration=2.134892071 podStartE2EDuration="4.519937803s" totalImagesPullingTime="2.385045732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:11:44.519477915 +0000 UTC m=+49.773267914" watchObservedRunningTime="2026-07-02 23:11:44.519937803 +0000 UTC m=+49.773727802"
Jul 02 23:11:45.456816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:45.456772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:11:45.457027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:45.456878    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:11:45.457027 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:45.456940    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 02 23:11:45.457027 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:45.456968    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7df9ff9cd7-4fn8r: secret "image-registry-tls" not found
Jul 02 23:11:45.457027 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:45.457010    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 02 23:11:45.457196 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:45.457041    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls podName:353a13fa-fc42-4342-a702-35ac283cd9b2 nodeName:}" failed. No retries permitted until 2026-07-02 23:12:01.457019338 +0000 UTC m=+66.710809317 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls") pod "image-registry-7df9ff9cd7-4fn8r" (UID: "353a13fa-fc42-4342-a702-35ac283cd9b2") : secret "image-registry-tls" not found
Jul 02 23:11:45.457196 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:45.457058    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls podName:d07d35c0-4de5-48b9-b2c2-890e8f479e17 nodeName:}" failed. No retries permitted until 2026-07-02 23:12:01.45705135 +0000 UTC m=+66.710841327 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls") pod "dns-default-lhb24" (UID: "d07d35c0-4de5-48b9-b2c2-890e8f479e17") : secret "dns-default-metrics-tls" not found
Jul 02 23:11:46.262441 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:46.262403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98bf3fef-ce15-4556-abac-89874cf50e09-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:46.262441 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:46.262445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:46.262891 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:46.262551    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 02 23:11:46.262891 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:11:46.262611    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls podName:98bf3fef-ce15-4556-abac-89874cf50e09 nodeName:}" failed. No retries permitted until 2026-07-02 23:11:54.262592328 +0000 UTC m=+59.516382317 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f7h7z" (UID: "98bf3fef-ce15-4556-abac-89874cf50e09") : secret "insights-runtime-extractor-tls" not found
Jul 02 23:11:46.262975 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:46.262898    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/98bf3fef-ce15-4556-abac-89874cf50e09-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:54.329625 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:54.329585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:54.332076 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:54.332054    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/98bf3fef-ce15-4556-abac-89874cf50e09-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f7h7z\" (UID: \"98bf3fef-ce15-4556-abac-89874cf50e09\") " pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:54.421486 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:54.421441    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f7h7z"
Jul 02 23:11:54.579536 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:54.579507    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f7h7z"]
Jul 02 23:11:54.582355 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:11:54.582328    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98bf3fef_ce15_4556_abac_89874cf50e09.slice/crio-6d12b8327771124a66be377b08f8a65b532e6acf08ca6a614788d52a79c1e6b0 WatchSource:0}: Error finding container 6d12b8327771124a66be377b08f8a65b532e6acf08ca6a614788d52a79c1e6b0: Status 404 returned error can't find the container with id 6d12b8327771124a66be377b08f8a65b532e6acf08ca6a614788d52a79c1e6b0
Jul 02 23:11:55.440486 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:55.440446    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9xtgc"
Jul 02 23:11:55.521422 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:55.521381    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f7h7z" event={"ID":"98bf3fef-ce15-4556-abac-89874cf50e09","Type":"ContainerStarted","Data":"6d12b8327771124a66be377b08f8a65b532e6acf08ca6a614788d52a79c1e6b0"}
Jul 02 23:11:58.531238 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:58.531202    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f7h7z" event={"ID":"98bf3fef-ce15-4556-abac-89874cf50e09","Type":"ContainerStarted","Data":"1652eafaa23b3e9c1dc36b75a215ef8623129e35b1015311a09dedd1916b6692"}
Jul 02 23:11:59.536206 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:11:59.536154    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f7h7z" event={"ID":"98bf3fef-ce15-4556-abac-89874cf50e09","Type":"ContainerStarted","Data":"4ee0a437552f8ae53943db40122efe53836f9d181814b4ba3bdf9b67c5053c8c"}
Jul 02 23:12:00.987719 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:00.987666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:12:00.988139 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:00.987734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs\") pod \"network-metrics-daemon-hjstr\" (UID: \"d7a0204c-3f9f-4f10-ace5-def5e62925dd\") " pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:12:00.991357 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:00.991326    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 02 23:12:00.991504 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:00.991449    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 02 23:12:01.000622 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.000596    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a0204c-3f9f-4f10-ace5-def5e62925dd-metrics-certs\") pod \"network-metrics-daemon-hjstr\" (UID: \"d7a0204c-3f9f-4f10-ace5-def5e62925dd\") " pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:12:01.000753 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.000715    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1d3f804-fd8f-43fa-81ab-82a96cc020e0-original-pull-secret\") pod \"global-pull-secret-syncer-vdlx9\" (UID: \"e1d3f804-fd8f-43fa-81ab-82a96cc020e0\") " pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:12:01.189209 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.189159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5kd\" (UniqueName: \"kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd\") pod \"network-check-target-l9lgl\" (UID: \"8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd\") " pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:12:01.192518 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.192478    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 02 23:12:01.202562 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.202528    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 02 23:12:01.212840 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.212810    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn5kd\" (UniqueName: \"kubernetes.io/projected/8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd-kube-api-access-cn5kd\") pod \"network-check-target-l9lgl\" (UID: \"8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd\") " pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:12:01.292874 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.292788    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-872wr\""
Jul 02 23:12:01.298375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.298349    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vdlx9"
Jul 02 23:12:01.300496 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.300464    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjstr"
Jul 02 23:12:01.306390 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.306367    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qmzc5\""
Jul 02 23:12:01.314807 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.314778    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:12:01.491464 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.491434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:12:01.491633 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.491480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:12:01.493757 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.493726    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d07d35c0-4de5-48b9-b2c2-890e8f479e17-metrics-tls\") pod \"dns-default-lhb24\" (UID: \"d07d35c0-4de5-48b9-b2c2-890e8f479e17\") " pod="openshift-dns/dns-default-lhb24"
Jul 02 23:12:01.493892 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.493867    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls\") pod \"image-registry-7df9ff9cd7-4fn8r\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") " pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:12:01.682151 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.682114    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kcbnf\""
Jul 02 23:12:01.682246 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.682223    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l9lgl"]
Jul 02 23:12:01.684607 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:12:01.684583    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e9c20fd_1d31_4602_9d01_2ad95ba7ddbd.slice/crio-6789453579be5e61e01444661d4d0b1dd4c2aa99a5a1f7c26ba7af981facdb8c WatchSource:0}: Error finding container 6789453579be5e61e01444661d4d0b1dd4c2aa99a5a1f7c26ba7af981facdb8c: Status 404 returned error can't find the container with id 6789453579be5e61e01444661d4d0b1dd4c2aa99a5a1f7c26ba7af981facdb8c
Jul 02 23:12:01.690150 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.690113    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:12:01.701293 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.701270    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2clb5\""
Jul 02 23:12:01.709712 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.709686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lhb24"
Jul 02 23:12:01.841505 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.841435    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"]
Jul 02 23:12:01.908434 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.908396    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hjstr"]
Jul 02 23:12:01.908957 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:12:01.908928    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353a13fa_fc42_4342_a702_35ac283cd9b2.slice/crio-899000020a8f66c02775d990df805d67d83bacf7fc5bf6b31c1dc4c75f10e14b WatchSource:0}: Error finding container 899000020a8f66c02775d990df805d67d83bacf7fc5bf6b31c1dc4c75f10e14b: Status 404 returned error can't find the container with id 899000020a8f66c02775d990df805d67d83bacf7fc5bf6b31c1dc4c75f10e14b
Jul 02 23:12:01.912652 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:12:01.912626    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7a0204c_3f9f_4f10_ace5_def5e62925dd.slice/crio-5634de9c5819ca36d9638ada940c6596e75b826843853cef0da2b4245b6c5407 WatchSource:0}: Error finding container 5634de9c5819ca36d9638ada940c6596e75b826843853cef0da2b4245b6c5407: Status 404 returned error can't find the container with id 5634de9c5819ca36d9638ada940c6596e75b826843853cef0da2b4245b6c5407
Jul 02 23:12:01.913171 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:01.913148    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vdlx9"]
Jul 02 23:12:01.916466 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:12:01.916447    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d3f804_fd8f_43fa_81ab_82a96cc020e0.slice/crio-f7623472192a4596f274c7c886d905bc556b97113d30caa8c2d7f73b82322bad WatchSource:0}: Error finding container f7623472192a4596f274c7c886d905bc556b97113d30caa8c2d7f73b82322bad: Status 404 returned error can't find the container with id f7623472192a4596f274c7c886d905bc556b97113d30caa8c2d7f73b82322bad
Jul 02 23:12:02.025073 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.025045    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869db98b7b-gxkfz"]
Jul 02 23:12:02.048171 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.048141    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"]
Jul 02 23:12:02.048395 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.048245    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869db98b7b-gxkfz"
Jul 02 23:12:02.051573 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.051552    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 02 23:12:02.052498 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.052331    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 02 23:12:02.052498 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.052391    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jhx22\""
Jul 02 23:12:02.052498 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.052450    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 02 23:12:02.052690 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.052507    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 02 23:12:02.070423 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.070401    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"]
Jul 02 23:12:02.070536 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.070512    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.075425 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.075399    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 02 23:12:02.075542 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.075434    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 02 23:12:02.075542 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.075454    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 02 23:12:02.075658 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.075591    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 02 23:12:02.076375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.076354    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"]
Jul 02 23:12:02.077104 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.077085    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869db98b7b-gxkfz"]
Jul 02 23:12:02.098633 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.098553    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc7070ab-4c07-46bf-8a72-b384dc2dc925-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-869db98b7b-gxkfz\" (UID: \"dc7070ab-4c07-46bf-8a72-b384dc2dc925\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869db98b7b-gxkfz"
Jul 02 23:12:02.098633 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.098584    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fb41a67-e94e-4ba2-8519-2c0081c61d39-additional-service-ca\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.098633 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.098603    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3fb41a67-e94e-4ba2-8519-2c0081c61d39-ca\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.098633 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.098628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tbng\" (UniqueName: \"kubernetes.io/projected/3fb41a67-e94e-4ba2-8519-2c0081c61d39-kube-api-access-9tbng\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.098858 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.098678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt57f\" (UniqueName: \"kubernetes.io/projected/dc7070ab-4c07-46bf-8a72-b384dc2dc925-kube-api-access-dt57f\") pod \"managed-serviceaccount-addon-agent-869db98b7b-gxkfz\" (UID: \"dc7070ab-4c07-46bf-8a72-b384dc2dc925\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869db98b7b-gxkfz"
Jul 02 23:12:02.098858 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.098699    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3fb41a67-e94e-4ba2-8519-2c0081c61d39-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.098858 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.098759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3fb41a67-e94e-4ba2-8519-2c0081c61d39-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.098858 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.098798    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3fb41a67-e94e-4ba2-8519-2c0081c61d39-hub\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.116517 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.116485    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lhb24"]
Jul 02 23:12:02.121295 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:12:02.121269    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd07d35c0_4de5_48b9_b2c2_890e8f479e17.slice/crio-e2244c73869537a5ab18305ca3ddc8e112799d14c5568244c77f2720381b4365 WatchSource:0}: Error finding container e2244c73869537a5ab18305ca3ddc8e112799d14c5568244c77f2720381b4365: Status 404 returned error can't find the container with id e2244c73869537a5ab18305ca3ddc8e112799d14c5568244c77f2720381b4365
Jul 02 23:12:02.129387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.129365    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6bf947ff87-xrbkj"]
Jul 02 23:12:02.155979 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.155953    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bf947ff87-xrbkj"]
Jul 02 23:12:02.156142 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.156067    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.199750 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.199719    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk2gp\" (UniqueName: \"kubernetes.io/projected/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-kube-api-access-xk2gp\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.199916 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.199755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt57f\" (UniqueName: \"kubernetes.io/projected/dc7070ab-4c07-46bf-8a72-b384dc2dc925-kube-api-access-dt57f\") pod \"managed-serviceaccount-addon-agent-869db98b7b-gxkfz\" (UID: \"dc7070ab-4c07-46bf-8a72-b384dc2dc925\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869db98b7b-gxkfz"
Jul 02 23:12:02.199916 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.199776    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-installation-pull-secrets\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.199916 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.199796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3fb41a67-e94e-4ba2-8519-2c0081c61d39-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.199916 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.199814    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-registry-tls\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.199916 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.199862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-bound-sa-token\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.200195 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.199943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3fb41a67-e94e-4ba2-8519-2c0081c61d39-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.200195 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.199968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3fb41a67-e94e-4ba2-8519-2c0081c61d39-hub\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.200195 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.199993    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-registry-certificates\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.200195 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.200028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc7070ab-4c07-46bf-8a72-b384dc2dc925-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-869db98b7b-gxkfz\" (UID: \"dc7070ab-4c07-46bf-8a72-b384dc2dc925\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869db98b7b-gxkfz"
Jul 02 23:12:02.200195 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.200053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fb41a67-e94e-4ba2-8519-2c0081c61d39-additional-service-ca\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.200195 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.200077    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-trusted-ca\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.200195 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.200104    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-image-registry-private-configuration\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.200195 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.200144    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3fb41a67-e94e-4ba2-8519-2c0081c61d39-ca\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.200195 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.200165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-ca-trust-extracted\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.200195 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.200195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tbng\" (UniqueName: \"kubernetes.io/projected/3fb41a67-e94e-4ba2-8519-2c0081c61d39-kube-api-access-9tbng\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.200844 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.200816    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fb41a67-e94e-4ba2-8519-2c0081c61d39-additional-service-ca\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.202782 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.202755    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3fb41a67-e94e-4ba2-8519-2c0081c61d39-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.202992 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.202966    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3fb41a67-e94e-4ba2-8519-2c0081c61d39-ca\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.203069 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.203005    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3fb41a67-e94e-4ba2-8519-2c0081c61d39-hub\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.203069 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.203015    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3fb41a67-e94e-4ba2-8519-2c0081c61d39-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.203756 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.203736    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dc7070ab-4c07-46bf-8a72-b384dc2dc925-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-869db98b7b-gxkfz\" (UID: \"dc7070ab-4c07-46bf-8a72-b384dc2dc925\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869db98b7b-gxkfz"
Jul 02 23:12:02.211638 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.211612    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tbng\" (UniqueName: \"kubernetes.io/projected/3fb41a67-e94e-4ba2-8519-2c0081c61d39-kube-api-access-9tbng\") pod \"cluster-proxy-proxy-agent-67df974948-jrq4q\" (UID: \"3fb41a67-e94e-4ba2-8519-2c0081c61d39\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.212000 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.211962    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt57f\" (UniqueName: \"kubernetes.io/projected/dc7070ab-4c07-46bf-8a72-b384dc2dc925-kube-api-access-dt57f\") pod \"managed-serviceaccount-addon-agent-869db98b7b-gxkfz\" (UID: \"dc7070ab-4c07-46bf-8a72-b384dc2dc925\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869db98b7b-gxkfz"
Jul 02 23:12:02.300945 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.300910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-registry-tls\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.300945 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.300960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-bound-sa-token\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.301378 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.301005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-registry-certificates\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.301378 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.301031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-trusted-ca\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.301378 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.301052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-image-registry-private-configuration\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.301378 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.301078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-ca-trust-extracted\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.301378 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.301131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk2gp\" (UniqueName: \"kubernetes.io/projected/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-kube-api-access-xk2gp\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.301378 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.301163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-installation-pull-secrets\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.301749 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.301717    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-ca-trust-extracted\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.302177 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.302091    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-registry-certificates\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.302177 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.302101    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-trusted-ca\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.304474 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.304446    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-installation-pull-secrets\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.304583 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.304494    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-registry-tls\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.304583 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.304519    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-image-registry-private-configuration\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.317507 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.317482    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-bound-sa-token\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.317610 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.317528    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk2gp\" (UniqueName: \"kubernetes.io/projected/c89ddf32-dea8-4da4-a6bc-738cafaa2bda-kube-api-access-xk2gp\") pod \"image-registry-6bf947ff87-xrbkj\" (UID: \"c89ddf32-dea8-4da4-a6bc-738cafaa2bda\") " pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.367806 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.367665    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869db98b7b-gxkfz"
Jul 02 23:12:02.379372 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.379188    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"
Jul 02 23:12:02.489618 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.489340    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:02.558855 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.558653    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r" event={"ID":"353a13fa-fc42-4342-a702-35ac283cd9b2","Type":"ContainerStarted","Data":"7bc2e5651cdbaec0386b2483c18b8d1cd8d319bc0f1401ac9aa3c3e5f7f0d9bc"}
Jul 02 23:12:02.558855 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.558711    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r" event={"ID":"353a13fa-fc42-4342-a702-35ac283cd9b2","Type":"ContainerStarted","Data":"899000020a8f66c02775d990df805d67d83bacf7fc5bf6b31c1dc4c75f10e14b"}
Jul 02 23:12:02.558855 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.558758    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:12:02.571077 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.570960    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lhb24" event={"ID":"d07d35c0-4de5-48b9-b2c2-890e8f479e17","Type":"ContainerStarted","Data":"e2244c73869537a5ab18305ca3ddc8e112799d14c5568244c77f2720381b4365"}
Jul 02 23:12:02.572661 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.572585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vdlx9" event={"ID":"e1d3f804-fd8f-43fa-81ab-82a96cc020e0","Type":"ContainerStarted","Data":"f7623472192a4596f274c7c886d905bc556b97113d30caa8c2d7f73b82322bad"}
Jul 02 23:12:02.574038 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.573970    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hjstr" event={"ID":"d7a0204c-3f9f-4f10-ace5-def5e62925dd","Type":"ContainerStarted","Data":"5634de9c5819ca36d9638ada940c6596e75b826843853cef0da2b4245b6c5407"}
Jul 02 23:12:02.581254 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.580568    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f7h7z" event={"ID":"98bf3fef-ce15-4556-abac-89874cf50e09","Type":"ContainerStarted","Data":"9aa22fa5ae576cd9ac7dd4c999dc8490bdf9684e22621f9ba8126e33ca9be1ab"}
Jul 02 23:12:02.584474 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.584416    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l9lgl" event={"ID":"8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd","Type":"ContainerStarted","Data":"6789453579be5e61e01444661d4d0b1dd4c2aa99a5a1f7c26ba7af981facdb8c"}
Jul 02 23:12:02.589098 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.588887    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r" podStartSLOduration=36.588867997 podStartE2EDuration="36.588867997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:02.588081227 +0000 UTC m=+67.841871229" watchObservedRunningTime="2026-07-02 23:12:02.588867997 +0000 UTC m=+67.842657998"
Jul 02 23:12:02.604510 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.604436    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869db98b7b-gxkfz"]
Jul 02 23:12:02.608628 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:12:02.608594    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc7070ab_4c07_46bf_8a72_b384dc2dc925.slice/crio-33d01e9aa89a7d30d01348aae95a0d86100a70c42eacb545301a9c005b0c5a81 WatchSource:0}: Error finding container 33d01e9aa89a7d30d01348aae95a0d86100a70c42eacb545301a9c005b0c5a81: Status 404 returned error can't find the container with id 33d01e9aa89a7d30d01348aae95a0d86100a70c42eacb545301a9c005b0c5a81
Jul 02 23:12:02.612071 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.611861    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-f7h7z" podStartSLOduration=17.36131769 podStartE2EDuration="24.611816158s" totalImagesPullingTime="7.250498468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:11:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:02.610277947 +0000 UTC m=+67.864067956" watchObservedRunningTime="2026-07-02 23:12:02.611816158 +0000 UTC m=+67.865606150"
Jul 02 23:12:02.634107 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.632383    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q"]
Jul 02 23:12:02.639205 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:12:02.639167    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fb41a67_e94e_4ba2_8519_2c0081c61d39.slice/crio-e54758904c9c63662cf5225b8d3c9bff15041ba50787bf349c88e4b746fa870e WatchSource:0}: Error finding container e54758904c9c63662cf5225b8d3c9bff15041ba50787bf349c88e4b746fa870e: Status 404 returned error can't find the container with id e54758904c9c63662cf5225b8d3c9bff15041ba50787bf349c88e4b746fa870e
Jul 02 23:12:02.737393 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:02.736208    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bf947ff87-xrbkj"]
Jul 02 23:12:03.591879 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:03.591828    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q" event={"ID":"3fb41a67-e94e-4ba2-8519-2c0081c61d39","Type":"ContainerStarted","Data":"e54758904c9c63662cf5225b8d3c9bff15041ba50787bf349c88e4b746fa870e"}
Jul 02 23:12:03.595308 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:03.594311    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj" event={"ID":"c89ddf32-dea8-4da4-a6bc-738cafaa2bda","Type":"ContainerStarted","Data":"f05b5c8d704dfb4428f98e2bdd524817f9c1b5e47e5acec8bd6d25ec8a1846c8"}
Jul 02 23:12:03.595308 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:03.594368    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj" event={"ID":"c89ddf32-dea8-4da4-a6bc-738cafaa2bda","Type":"ContainerStarted","Data":"2f33f7e8ea8ace4bbc9087459911b4aa8d85118b311ede6d349e65aa676e2da3"}
Jul 02 23:12:03.595308 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:03.595252    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:03.598058 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:03.597810    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869db98b7b-gxkfz" event={"ID":"dc7070ab-4c07-46bf-8a72-b384dc2dc925","Type":"ContainerStarted","Data":"33d01e9aa89a7d30d01348aae95a0d86100a70c42eacb545301a9c005b0c5a81"}
Jul 02 23:12:03.616515 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:03.616455    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj" podStartSLOduration=1.616412345 podStartE2EDuration="1.616412345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:12:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:03.614233066 +0000 UTC m=+68.868023066" watchObservedRunningTime="2026-07-02 23:12:03.616412345 +0000 UTC m=+68.870202324"
Jul 02 23:12:10.640770 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.640731    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q" event={"ID":"3fb41a67-e94e-4ba2-8519-2c0081c61d39","Type":"ContainerStarted","Data":"84486bc35b70019ffcf5d88319595f1fa84da59c08909744ef342625aaf636b7"}
Jul 02 23:12:10.641264 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.640785    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q" event={"ID":"3fb41a67-e94e-4ba2-8519-2c0081c61d39","Type":"ContainerStarted","Data":"c6367a733589c0769cd139915596fefb75e7e98d2a0d5b78943233536b12556f"}
Jul 02 23:12:10.641264 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.640805    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q" event={"ID":"3fb41a67-e94e-4ba2-8519-2c0081c61d39","Type":"ContainerStarted","Data":"d2c8d1a56f3618e3b64545ba64002ed6e16e5af02533fc999d35aa7e35d024bc"}
Jul 02 23:12:10.642483 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.642461    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lhb24" event={"ID":"d07d35c0-4de5-48b9-b2c2-890e8f479e17","Type":"ContainerStarted","Data":"c18efc560ce076b9cc79bc02f90cdeba7bb7b817f32cec7da152b77fc415af92"}
Jul 02 23:12:10.642591 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.642502    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lhb24" event={"ID":"d07d35c0-4de5-48b9-b2c2-890e8f479e17","Type":"ContainerStarted","Data":"ace010ff4c06a1421a59d91a60848012a330a6cc45afd23aeabd67596a64f8bb"}
Jul 02 23:12:10.642591 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.642546    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lhb24"
Jul 02 23:12:10.643778 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.643755    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vdlx9" event={"ID":"e1d3f804-fd8f-43fa-81ab-82a96cc020e0","Type":"ContainerStarted","Data":"128b87c17f6a666a1a339a3aabde72caf875620f81e44ac5ed804b15e00bea89"}
Jul 02 23:12:10.645477 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.645457    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hjstr" event={"ID":"d7a0204c-3f9f-4f10-ace5-def5e62925dd","Type":"ContainerStarted","Data":"112e936192f980b3c718fe2aacf49c3656db25a2779c4446e78218940f38ad91"}
Jul 02 23:12:10.645554 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.645500    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hjstr" event={"ID":"d7a0204c-3f9f-4f10-ace5-def5e62925dd","Type":"ContainerStarted","Data":"5ab3cc96fdf67a038e816d34e9715f5550008657a0eafb5090c4b09de27c9780"}
Jul 02 23:12:10.646894 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.646872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l9lgl" event={"ID":"8e9c20fd-1d31-4602-9d01-2ad95ba7ddbd","Type":"ContainerStarted","Data":"5eb7e99a297507d9f5989a3a12360586c68b10498827d8d5f13dc143324d26de"}
Jul 02 23:12:10.646987 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.646972    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:12:10.648189 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.648166    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869db98b7b-gxkfz" event={"ID":"dc7070ab-4c07-46bf-8a72-b384dc2dc925","Type":"ContainerStarted","Data":"75f7b873a3334d99aebaab84b637f1b4a3d1f143fcaffd5ebfe5ac33093e659d"}
Jul 02 23:12:10.660301 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.660250    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67df974948-jrq4q" podStartSLOduration=2.393051163 podStartE2EDuration="9.660238053s" totalImagesPullingTime="7.26718689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:10.658797848 +0000 UTC m=+75.912587848" watchObservedRunningTime="2026-07-02 23:12:10.660238053 +0000 UTC m=+75.914028051"
Jul 02 23:12:10.695506 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.695440    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lhb24" podStartSLOduration=33.960937854 podStartE2EDuration="41.695419465s" totalImagesPullingTime="7.734481611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:10.695184775 +0000 UTC m=+75.948974774" watchObservedRunningTime="2026-07-02 23:12:10.695419465 +0000 UTC m=+75.949209465"
Jul 02 23:12:10.696149 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.696107    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l9lgl" podStartSLOduration=66.525092024 podStartE2EDuration="1m14.696096506s" totalImagesPullingTime="8.171004482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:10.676601734 +0000 UTC m=+75.930391735" watchObservedRunningTime="2026-07-02 23:12:10.696096506 +0000 UTC m=+75.949886483"
Jul 02 23:12:10.709506 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.709465    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-869db98b7b-gxkfz" podStartSLOduration=2.445197293 podStartE2EDuration="9.709452383s" totalImagesPullingTime="7.26425509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:12:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:10.708580179 +0000 UTC m=+75.962370178" watchObservedRunningTime="2026-07-02 23:12:10.709452383 +0000 UTC m=+75.963242406"
Jul 02 23:12:10.725530 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.725491    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hjstr" podStartSLOduration=67.809101195 podStartE2EDuration="1m15.725478614s" totalImagesPullingTime="7.916377419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:10:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:10.724514808 +0000 UTC m=+75.978304806" watchObservedRunningTime="2026-07-02 23:12:10.725478614 +0000 UTC m=+75.979268591"
Jul 02 23:12:10.738419 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:10.738357    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vdlx9" podStartSLOduration=66.790117025 podStartE2EDuration="1m14.738344479s" totalImagesPullingTime="7.948227454s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:10.73795402 +0000 UTC m=+75.991744030" watchObservedRunningTime="2026-07-02 23:12:10.738344479 +0000 UTC m=+75.992134479"
Jul 02 23:12:18.731384 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.731249    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-n8wxc"]
Jul 02 23:12:18.735708 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.735685    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.742197 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.742169    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 02 23:12:18.745327 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.745305    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 02 23:12:18.745609 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.745499    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 02 23:12:18.745695 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.745449    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 02 23:12:18.745695 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.745655    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wq2d5\""
Jul 02 23:12:18.746044 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.745384    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 02 23:12:18.746044 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.745821    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 02 23:12:18.837866 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.837832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cgzg\" (UniqueName: \"kubernetes.io/projected/a4881e10-2a01-489c-ae57-c0de17e56bd5-kube-api-access-6cgzg\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.837866 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.837878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a4881e10-2a01-489c-ae57-c0de17e56bd5-root\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.838111 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.838000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4881e10-2a01-489c-ae57-c0de17e56bd5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.838111 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.838039    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4881e10-2a01-489c-ae57-c0de17e56bd5-metrics-client-ca\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.838111 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.838072    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a4881e10-2a01-489c-ae57-c0de17e56bd5-node-exporter-accelerators-collector-config\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.838269 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.838129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a4881e10-2a01-489c-ae57-c0de17e56bd5-node-exporter-textfile\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.838269 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.838172    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a4881e10-2a01-489c-ae57-c0de17e56bd5-node-exporter-wtmp\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.838269 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.838235    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4881e10-2a01-489c-ae57-c0de17e56bd5-sys\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.838378 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.838277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a4881e10-2a01-489c-ae57-c0de17e56bd5-node-exporter-tls\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.939333 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.939298    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4881e10-2a01-489c-ae57-c0de17e56bd5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.939333 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.939343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4881e10-2a01-489c-ae57-c0de17e56bd5-metrics-client-ca\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.939577 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.939370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a4881e10-2a01-489c-ae57-c0de17e56bd5-node-exporter-accelerators-collector-config\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.939577 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.939408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a4881e10-2a01-489c-ae57-c0de17e56bd5-node-exporter-textfile\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.939577 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.939433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a4881e10-2a01-489c-ae57-c0de17e56bd5-node-exporter-wtmp\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.939577 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.939472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4881e10-2a01-489c-ae57-c0de17e56bd5-sys\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.939577 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.939500    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a4881e10-2a01-489c-ae57-c0de17e56bd5-node-exporter-tls\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.939577 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.939540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cgzg\" (UniqueName: \"kubernetes.io/projected/a4881e10-2a01-489c-ae57-c0de17e56bd5-kube-api-access-6cgzg\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.939577 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.939567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a4881e10-2a01-489c-ae57-c0de17e56bd5-root\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.939907 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.939649    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a4881e10-2a01-489c-ae57-c0de17e56bd5-root\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.939907 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.939769    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a4881e10-2a01-489c-ae57-c0de17e56bd5-node-exporter-wtmp\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.939907 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.939809    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4881e10-2a01-489c-ae57-c0de17e56bd5-sys\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.940058 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.940007    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a4881e10-2a01-489c-ae57-c0de17e56bd5-node-exporter-textfile\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.940058 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.940035    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4881e10-2a01-489c-ae57-c0de17e56bd5-metrics-client-ca\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.940176 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.940059    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a4881e10-2a01-489c-ae57-c0de17e56bd5-node-exporter-accelerators-collector-config\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.941872 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.941847    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4881e10-2a01-489c-ae57-c0de17e56bd5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.941999 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.941983    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a4881e10-2a01-489c-ae57-c0de17e56bd5-node-exporter-tls\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:18.949997 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:18.949935    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cgzg\" (UniqueName: \"kubernetes.io/projected/a4881e10-2a01-489c-ae57-c0de17e56bd5-kube-api-access-6cgzg\") pod \"node-exporter-n8wxc\" (UID: \"a4881e10-2a01-489c-ae57-c0de17e56bd5\") " pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:19.049231 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:19.049143    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n8wxc"
Jul 02 23:12:19.059979 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:12:19.059948    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4881e10_2a01_489c_ae57_c0de17e56bd5.slice/crio-b6425412fcc9341d39ec6bbcfcd2def914c8b37f55c70c09b088fb78f61a52ab WatchSource:0}: Error finding container b6425412fcc9341d39ec6bbcfcd2def914c8b37f55c70c09b088fb78f61a52ab: Status 404 returned error can't find the container with id b6425412fcc9341d39ec6bbcfcd2def914c8b37f55c70c09b088fb78f61a52ab
Jul 02 23:12:19.683183 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:19.683136    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n8wxc" event={"ID":"a4881e10-2a01-489c-ae57-c0de17e56bd5","Type":"ContainerStarted","Data":"b6425412fcc9341d39ec6bbcfcd2def914c8b37f55c70c09b088fb78f61a52ab"}
Jul 02 23:12:20.653791 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:20.653759    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lhb24"
Jul 02 23:12:20.688892 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:20.688862    2580 generic.go:356] "Generic (PLEG): container finished" podID="a4881e10-2a01-489c-ae57-c0de17e56bd5" containerID="279ebe39ced3c708fd381c307cdfd1b3ebf112fa57921b5481a961a8b4d89f2e" exitCode=0
Jul 02 23:12:20.689048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:20.688934    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n8wxc" event={"ID":"a4881e10-2a01-489c-ae57-c0de17e56bd5","Type":"ContainerDied","Data":"279ebe39ced3c708fd381c307cdfd1b3ebf112fa57921b5481a961a8b4d89f2e"}
Jul 02 23:12:21.695543 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:21.695509    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n8wxc" event={"ID":"a4881e10-2a01-489c-ae57-c0de17e56bd5","Type":"ContainerStarted","Data":"0259705bd8658b3987f82df9bfaea4386672d1c6cea8e88f891fd8b2e0ae5071"}
Jul 02 23:12:21.695939 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:21.695564    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n8wxc" event={"ID":"a4881e10-2a01-489c-ae57-c0de17e56bd5","Type":"ContainerStarted","Data":"3a09471a044011e303e9d7c0389ddd987c529e7966766ad834f2219fad59a25b"}
Jul 02 23:12:21.717037 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:21.716975    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-n8wxc" podStartSLOduration=2.860532326 podStartE2EDuration="3.716959463s" totalImagesPullingTime="791.863537ms" totalInitContainerRuntime="64.5636ms" isStatefulPod=true podCreationTimestamp="2026-07-02 23:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:12:21.716005129 +0000 UTC m=+86.969795145" watchObservedRunningTime="2026-07-02 23:12:21.716959463 +0000 UTC m=+86.970749463"
Jul 02 23:12:22.494300 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:22.494266    2580 patch_prober.go:28] interesting pod/image-registry-6bf947ff87-xrbkj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 02 23:12:22.494470 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:22.494326    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj" podUID="c89ddf32-dea8-4da4-a6bc-738cafaa2bda" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 23:12:23.606985 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:23.606951    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:12:25.623584 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:25.623554    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6bf947ff87-xrbkj"
Jul 02 23:12:28.627620 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:28.627566    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r" podUID="353a13fa-fc42-4342-a702-35ac283cd9b2" containerName="registry" containerID="cri-o://7bc2e5651cdbaec0386b2483c18b8d1cd8d319bc0f1401ac9aa3c3e5f7f0d9bc" gracePeriod=30
Jul 02 23:12:28.865736 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:28.865712    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:12:29.018778 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.018741    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/353a13fa-fc42-4342-a702-35ac283cd9b2-image-registry-private-configuration\") pod \"353a13fa-fc42-4342-a702-35ac283cd9b2\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") "
Jul 02 23:12:29.018778 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.018782    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls\") pod \"353a13fa-fc42-4342-a702-35ac283cd9b2\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") "
Jul 02 23:12:29.019026 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.018798    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-bound-sa-token\") pod \"353a13fa-fc42-4342-a702-35ac283cd9b2\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") "
Jul 02 23:12:29.019026 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.018837    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/353a13fa-fc42-4342-a702-35ac283cd9b2-trusted-ca\") pod \"353a13fa-fc42-4342-a702-35ac283cd9b2\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") "
Jul 02 23:12:29.019026 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.018857    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/353a13fa-fc42-4342-a702-35ac283cd9b2-ca-trust-extracted\") pod \"353a13fa-fc42-4342-a702-35ac283cd9b2\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") "
Jul 02 23:12:29.019026 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.018883    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frjd5\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-kube-api-access-frjd5\") pod \"353a13fa-fc42-4342-a702-35ac283cd9b2\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") "
Jul 02 23:12:29.019026 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.018925    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/353a13fa-fc42-4342-a702-35ac283cd9b2-installation-pull-secrets\") pod \"353a13fa-fc42-4342-a702-35ac283cd9b2\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") "
Jul 02 23:12:29.019301 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.019277    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-certificates\") pod \"353a13fa-fc42-4342-a702-35ac283cd9b2\" (UID: \"353a13fa-fc42-4342-a702-35ac283cd9b2\") "
Jul 02 23:12:29.019573 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.019419    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/353a13fa-fc42-4342-a702-35ac283cd9b2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "353a13fa-fc42-4342-a702-35ac283cd9b2" (UID: "353a13fa-fc42-4342-a702-35ac283cd9b2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 23:12:29.019573 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.019542    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/353a13fa-fc42-4342-a702-35ac283cd9b2-trusted-ca\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:12:29.019843 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.019796    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "353a13fa-fc42-4342-a702-35ac283cd9b2" (UID: "353a13fa-fc42-4342-a702-35ac283cd9b2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 23:12:29.021487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.021454    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-kube-api-access-frjd5" (OuterVolumeSpecName: "kube-api-access-frjd5") pod "353a13fa-fc42-4342-a702-35ac283cd9b2" (UID: "353a13fa-fc42-4342-a702-35ac283cd9b2"). InnerVolumeSpecName "kube-api-access-frjd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:12:29.021612 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.021530    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/353a13fa-fc42-4342-a702-35ac283cd9b2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "353a13fa-fc42-4342-a702-35ac283cd9b2" (UID: "353a13fa-fc42-4342-a702-35ac283cd9b2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 23:12:29.021612 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.021568    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "353a13fa-fc42-4342-a702-35ac283cd9b2" (UID: "353a13fa-fc42-4342-a702-35ac283cd9b2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:12:29.021744 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.021726    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/353a13fa-fc42-4342-a702-35ac283cd9b2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "353a13fa-fc42-4342-a702-35ac283cd9b2" (UID: "353a13fa-fc42-4342-a702-35ac283cd9b2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 23:12:29.021801 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.021744    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "353a13fa-fc42-4342-a702-35ac283cd9b2" (UID: "353a13fa-fc42-4342-a702-35ac283cd9b2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:12:29.029077 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.029046    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/353a13fa-fc42-4342-a702-35ac283cd9b2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "353a13fa-fc42-4342-a702-35ac283cd9b2" (UID: "353a13fa-fc42-4342-a702-35ac283cd9b2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:12:29.120068 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.120026    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/353a13fa-fc42-4342-a702-35ac283cd9b2-ca-trust-extracted\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:12:29.120068 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.120059    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frjd5\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-kube-api-access-frjd5\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:12:29.120068 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.120070    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/353a13fa-fc42-4342-a702-35ac283cd9b2-installation-pull-secrets\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:12:29.120068 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.120080    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-certificates\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:12:29.120337 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.120090    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/353a13fa-fc42-4342-a702-35ac283cd9b2-image-registry-private-configuration\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:12:29.120337 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.120099    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-registry-tls\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:12:29.120337 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.120107    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/353a13fa-fc42-4342-a702-35ac283cd9b2-bound-sa-token\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:12:29.725291 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.725261    2580 generic.go:356] "Generic (PLEG): container finished" podID="353a13fa-fc42-4342-a702-35ac283cd9b2" containerID="7bc2e5651cdbaec0386b2483c18b8d1cd8d319bc0f1401ac9aa3c3e5f7f0d9bc" exitCode=0
Jul 02 23:12:29.725678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.725308    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"
Jul 02 23:12:29.725678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.725312    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r" event={"ID":"353a13fa-fc42-4342-a702-35ac283cd9b2","Type":"ContainerDied","Data":"7bc2e5651cdbaec0386b2483c18b8d1cd8d319bc0f1401ac9aa3c3e5f7f0d9bc"}
Jul 02 23:12:29.725678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.725416    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7df9ff9cd7-4fn8r" event={"ID":"353a13fa-fc42-4342-a702-35ac283cd9b2","Type":"ContainerDied","Data":"899000020a8f66c02775d990df805d67d83bacf7fc5bf6b31c1dc4c75f10e14b"}
Jul 02 23:12:29.725678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.725437    2580 scope.go:117] "RemoveContainer" containerID="7bc2e5651cdbaec0386b2483c18b8d1cd8d319bc0f1401ac9aa3c3e5f7f0d9bc"
Jul 02 23:12:29.733560 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.733543    2580 scope.go:117] "RemoveContainer" containerID="7bc2e5651cdbaec0386b2483c18b8d1cd8d319bc0f1401ac9aa3c3e5f7f0d9bc"
Jul 02 23:12:29.733858 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:12:29.733835    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bc2e5651cdbaec0386b2483c18b8d1cd8d319bc0f1401ac9aa3c3e5f7f0d9bc\": container with ID starting with 7bc2e5651cdbaec0386b2483c18b8d1cd8d319bc0f1401ac9aa3c3e5f7f0d9bc not found: ID does not exist" containerID="7bc2e5651cdbaec0386b2483c18b8d1cd8d319bc0f1401ac9aa3c3e5f7f0d9bc"
Jul 02 23:12:29.733911 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.733868    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bc2e5651cdbaec0386b2483c18b8d1cd8d319bc0f1401ac9aa3c3e5f7f0d9bc"} err="failed to get container status \"7bc2e5651cdbaec0386b2483c18b8d1cd8d319bc0f1401ac9aa3c3e5f7f0d9bc\": rpc error: code = NotFound desc = could not find container \"7bc2e5651cdbaec0386b2483c18b8d1cd8d319bc0f1401ac9aa3c3e5f7f0d9bc\": container with ID starting with 7bc2e5651cdbaec0386b2483c18b8d1cd8d319bc0f1401ac9aa3c3e5f7f0d9bc not found: ID does not exist"
Jul 02 23:12:29.746739 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.746713    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"]
Jul 02 23:12:29.752359 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:29.752339    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7df9ff9cd7-4fn8r"]
Jul 02 23:12:31.281849 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:31.281808    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="353a13fa-fc42-4342-a702-35ac283cd9b2" path="/var/lib/kubelet/pods/353a13fa-fc42-4342-a702-35ac283cd9b2/volumes"
Jul 02 23:12:41.654328 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:12:41.654296    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l9lgl"
Jul 02 23:14:52.690900 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.690867    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"]
Jul 02 23:14:52.691334 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.691103    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="353a13fa-fc42-4342-a702-35ac283cd9b2" containerName="registry"
Jul 02 23:14:52.691334 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.691113    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="353a13fa-fc42-4342-a702-35ac283cd9b2" containerName="registry"
Jul 02 23:14:52.691334 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.691187    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="353a13fa-fc42-4342-a702-35ac283cd9b2" containerName="registry"
Jul 02 23:14:52.693074 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.693059    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"
Jul 02 23:14:52.695890 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.695868    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 02 23:14:52.696030 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.695986    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wdwfr\""
Jul 02 23:14:52.697021 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.697005    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 02 23:14:52.705405 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.705380    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"]
Jul 02 23:14:52.752274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.752234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/058ad30b-c436-4807-ba6d-a46ba427ed32-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55\" (UID: \"058ad30b-c436-4807-ba6d-a46ba427ed32\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"
Jul 02 23:14:52.752274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.752280    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv2lm\" (UniqueName: \"kubernetes.io/projected/058ad30b-c436-4807-ba6d-a46ba427ed32-kube-api-access-pv2lm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55\" (UID: \"058ad30b-c436-4807-ba6d-a46ba427ed32\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"
Jul 02 23:14:52.752505 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.752352    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/058ad30b-c436-4807-ba6d-a46ba427ed32-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55\" (UID: \"058ad30b-c436-4807-ba6d-a46ba427ed32\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"
Jul 02 23:14:52.853230 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.853197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/058ad30b-c436-4807-ba6d-a46ba427ed32-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55\" (UID: \"058ad30b-c436-4807-ba6d-a46ba427ed32\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"
Jul 02 23:14:52.853358 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.853241    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv2lm\" (UniqueName: \"kubernetes.io/projected/058ad30b-c436-4807-ba6d-a46ba427ed32-kube-api-access-pv2lm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55\" (UID: \"058ad30b-c436-4807-ba6d-a46ba427ed32\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"
Jul 02 23:14:52.853358 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.853287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/058ad30b-c436-4807-ba6d-a46ba427ed32-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55\" (UID: \"058ad30b-c436-4807-ba6d-a46ba427ed32\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"
Jul 02 23:14:52.853676 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.853652    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/058ad30b-c436-4807-ba6d-a46ba427ed32-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55\" (UID: \"058ad30b-c436-4807-ba6d-a46ba427ed32\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"
Jul 02 23:14:52.853749 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.853662    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/058ad30b-c436-4807-ba6d-a46ba427ed32-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55\" (UID: \"058ad30b-c436-4807-ba6d-a46ba427ed32\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"
Jul 02 23:14:52.866663 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:52.866638    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv2lm\" (UniqueName: \"kubernetes.io/projected/058ad30b-c436-4807-ba6d-a46ba427ed32-kube-api-access-pv2lm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55\" (UID: \"058ad30b-c436-4807-ba6d-a46ba427ed32\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"
Jul 02 23:14:53.002854 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:53.002760    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"
Jul 02 23:14:53.130907 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:53.130865    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"]
Jul 02 23:14:53.133749 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:14:53.133721    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod058ad30b_c436_4807_ba6d_a46ba427ed32.slice/crio-26f481add03e73f5f4b5f0f45172eb753eff0fb0d26a6ea0080bd84604674213 WatchSource:0}: Error finding container 26f481add03e73f5f4b5f0f45172eb753eff0fb0d26a6ea0080bd84604674213: Status 404 returned error can't find the container with id 26f481add03e73f5f4b5f0f45172eb753eff0fb0d26a6ea0080bd84604674213
Jul 02 23:14:53.194297 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:53.194256    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55" event={"ID":"058ad30b-c436-4807-ba6d-a46ba427ed32","Type":"ContainerStarted","Data":"26f481add03e73f5f4b5f0f45172eb753eff0fb0d26a6ea0080bd84604674213"}
Jul 02 23:14:59.218946 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:59.218911    2580 generic.go:356] "Generic (PLEG): container finished" podID="058ad30b-c436-4807-ba6d-a46ba427ed32" containerID="e4b02f50a581fef67ccb4eff88a2f202e6e9efd297ffef0f99ef68974485496f" exitCode=0
Jul 02 23:14:59.219373 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:14:59.219005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55" event={"ID":"058ad30b-c436-4807-ba6d-a46ba427ed32","Type":"ContainerDied","Data":"e4b02f50a581fef67ccb4eff88a2f202e6e9efd297ffef0f99ef68974485496f"}
Jul 02 23:15:02.230934 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:02.230899    2580 generic.go:356] "Generic (PLEG): container finished" podID="058ad30b-c436-4807-ba6d-a46ba427ed32" containerID="58d7da412e2d64e49e18af5e22e5cf4463b63f6c7cbada054ee7ef49e19938a7" exitCode=0
Jul 02 23:15:02.231364 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:02.230990    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55" event={"ID":"058ad30b-c436-4807-ba6d-a46ba427ed32","Type":"ContainerDied","Data":"58d7da412e2d64e49e18af5e22e5cf4463b63f6c7cbada054ee7ef49e19938a7"}
Jul 02 23:15:09.257689 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:09.257604    2580 generic.go:356] "Generic (PLEG): container finished" podID="058ad30b-c436-4807-ba6d-a46ba427ed32" containerID="720a75420d5ed749f3fba71d5fdaf6ecd4fcabd3f4597b9d8038d3f693754a91" exitCode=0
Jul 02 23:15:09.257689 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:09.257655    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55" event={"ID":"058ad30b-c436-4807-ba6d-a46ba427ed32","Type":"ContainerDied","Data":"720a75420d5ed749f3fba71d5fdaf6ecd4fcabd3f4597b9d8038d3f693754a91"}
Jul 02 23:15:10.384398 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:10.384367    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"
Jul 02 23:15:10.486934 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:10.486894    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/058ad30b-c436-4807-ba6d-a46ba427ed32-bundle\") pod \"058ad30b-c436-4807-ba6d-a46ba427ed32\" (UID: \"058ad30b-c436-4807-ba6d-a46ba427ed32\") "
Jul 02 23:15:10.486934 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:10.486942    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pv2lm\" (UniqueName: \"kubernetes.io/projected/058ad30b-c436-4807-ba6d-a46ba427ed32-kube-api-access-pv2lm\") pod \"058ad30b-c436-4807-ba6d-a46ba427ed32\" (UID: \"058ad30b-c436-4807-ba6d-a46ba427ed32\") "
Jul 02 23:15:10.487217 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:10.486987    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/058ad30b-c436-4807-ba6d-a46ba427ed32-util\") pod \"058ad30b-c436-4807-ba6d-a46ba427ed32\" (UID: \"058ad30b-c436-4807-ba6d-a46ba427ed32\") "
Jul 02 23:15:10.487426 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:10.487399    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/058ad30b-c436-4807-ba6d-a46ba427ed32-bundle" (OuterVolumeSpecName: "bundle") pod "058ad30b-c436-4807-ba6d-a46ba427ed32" (UID: "058ad30b-c436-4807-ba6d-a46ba427ed32"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:15:10.489207 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:10.489178    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/058ad30b-c436-4807-ba6d-a46ba427ed32-kube-api-access-pv2lm" (OuterVolumeSpecName: "kube-api-access-pv2lm") pod "058ad30b-c436-4807-ba6d-a46ba427ed32" (UID: "058ad30b-c436-4807-ba6d-a46ba427ed32"). InnerVolumeSpecName "kube-api-access-pv2lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:15:10.491968 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:10.491946    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/058ad30b-c436-4807-ba6d-a46ba427ed32-util" (OuterVolumeSpecName: "util") pod "058ad30b-c436-4807-ba6d-a46ba427ed32" (UID: "058ad30b-c436-4807-ba6d-a46ba427ed32"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:15:10.588340 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:10.588227    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv2lm\" (UniqueName: \"kubernetes.io/projected/058ad30b-c436-4807-ba6d-a46ba427ed32-kube-api-access-pv2lm\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:15:10.588340 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:10.588276    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/058ad30b-c436-4807-ba6d-a46ba427ed32-util\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:15:10.588340 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:10.588289    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/058ad30b-c436-4807-ba6d-a46ba427ed32-bundle\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:15:11.266070 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:11.266034    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55" event={"ID":"058ad30b-c436-4807-ba6d-a46ba427ed32","Type":"ContainerDied","Data":"26f481add03e73f5f4b5f0f45172eb753eff0fb0d26a6ea0080bd84604674213"}
Jul 02 23:15:11.266262 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:11.266103    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26f481add03e73f5f4b5f0f45172eb753eff0fb0d26a6ea0080bd84604674213"
Jul 02 23:15:11.266262 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:11.266050    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fx2x55"
Jul 02 23:15:35.146399 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.146363    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"]
Jul 02 23:15:35.146809 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.146619    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="058ad30b-c436-4807-ba6d-a46ba427ed32" containerName="util"
Jul 02 23:15:35.146809 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.146629    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="058ad30b-c436-4807-ba6d-a46ba427ed32" containerName="util"
Jul 02 23:15:35.146809 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.146641    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="058ad30b-c436-4807-ba6d-a46ba427ed32" containerName="extract"
Jul 02 23:15:35.146809 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.146646    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="058ad30b-c436-4807-ba6d-a46ba427ed32" containerName="extract"
Jul 02 23:15:35.146809 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.146660    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="058ad30b-c436-4807-ba6d-a46ba427ed32" containerName="pull"
Jul 02 23:15:35.146809 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.146665    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="058ad30b-c436-4807-ba6d-a46ba427ed32" containerName="pull"
Jul 02 23:15:35.146809 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.146708    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="058ad30b-c436-4807-ba6d-a46ba427ed32" containerName="extract"
Jul 02 23:15:35.148475 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.148458    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.153345 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.153319    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kfpkl\""
Jul 02 23:15:35.153720 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.153702    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 02 23:15:35.153953 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.153886    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 02 23:15:35.153953 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.153924    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 02 23:15:35.154352 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.154334    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 02 23:15:35.265545 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.265508    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfff93ba-acef-4836-be23-4663650be31e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d98bc9d58-svd4b\" (UID: \"cfff93ba-acef-4836-be23-4663650be31e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.265724 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.265562    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cfff93ba-acef-4836-be23-4663650be31e-tmp\") pod \"opendatahub-operator-controller-manager-7d98bc9d58-svd4b\" (UID: \"cfff93ba-acef-4836-be23-4663650be31e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.265724 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.265653    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kspc9\" (UniqueName: \"kubernetes.io/projected/cfff93ba-acef-4836-be23-4663650be31e-kube-api-access-kspc9\") pod \"opendatahub-operator-controller-manager-7d98bc9d58-svd4b\" (UID: \"cfff93ba-acef-4836-be23-4663650be31e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.265724 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.265702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cfff93ba-acef-4836-be23-4663650be31e-manifests\") pod \"opendatahub-operator-controller-manager-7d98bc9d58-svd4b\" (UID: \"cfff93ba-acef-4836-be23-4663650be31e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.265830 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.265752    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cfff93ba-acef-4836-be23-4663650be31e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d98bc9d58-svd4b\" (UID: \"cfff93ba-acef-4836-be23-4663650be31e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.366567 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.366533    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cfff93ba-acef-4836-be23-4663650be31e-tmp\") pod \"opendatahub-operator-controller-manager-7d98bc9d58-svd4b\" (UID: \"cfff93ba-acef-4836-be23-4663650be31e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.366717 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.366598    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kspc9\" (UniqueName: \"kubernetes.io/projected/cfff93ba-acef-4836-be23-4663650be31e-kube-api-access-kspc9\") pod \"opendatahub-operator-controller-manager-7d98bc9d58-svd4b\" (UID: \"cfff93ba-acef-4836-be23-4663650be31e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.366717 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.366653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cfff93ba-acef-4836-be23-4663650be31e-manifests\") pod \"opendatahub-operator-controller-manager-7d98bc9d58-svd4b\" (UID: \"cfff93ba-acef-4836-be23-4663650be31e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.366717 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.366697    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cfff93ba-acef-4836-be23-4663650be31e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d98bc9d58-svd4b\" (UID: \"cfff93ba-acef-4836-be23-4663650be31e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.366850 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.366730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfff93ba-acef-4836-be23-4663650be31e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d98bc9d58-svd4b\" (UID: \"cfff93ba-acef-4836-be23-4663650be31e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.367095 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.367063    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cfff93ba-acef-4836-be23-4663650be31e-manifests\") pod \"opendatahub-operator-controller-manager-7d98bc9d58-svd4b\" (UID: \"cfff93ba-acef-4836-be23-4663650be31e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.368764 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.368742    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cfff93ba-acef-4836-be23-4663650be31e-tmp\") pod \"opendatahub-operator-controller-manager-7d98bc9d58-svd4b\" (UID: \"cfff93ba-acef-4836-be23-4663650be31e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.369051 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.369034    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cfff93ba-acef-4836-be23-4663650be31e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d98bc9d58-svd4b\" (UID: \"cfff93ba-acef-4836-be23-4663650be31e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.369161 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.369143    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfff93ba-acef-4836-be23-4663650be31e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d98bc9d58-svd4b\" (UID: \"cfff93ba-acef-4836-be23-4663650be31e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.405663 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.405598    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kspc9\" (UniqueName: \"kubernetes.io/projected/cfff93ba-acef-4836-be23-4663650be31e-kube-api-access-kspc9\") pod \"opendatahub-operator-controller-manager-7d98bc9d58-svd4b\" (UID: \"cfff93ba-acef-4836-be23-4663650be31e\") " pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.458619 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.458585    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:35.482008 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.481972    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"]
Jul 02 23:15:35.625198 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:35.625166    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"]
Jul 02 23:15:35.635383 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:15:35.635345    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfff93ba_acef_4836_be23_4663650be31e.slice/crio-f938555efed2357a865a44498d42192e81efdb27f0fd67b13502eb673e19caf5 WatchSource:0}: Error finding container f938555efed2357a865a44498d42192e81efdb27f0fd67b13502eb673e19caf5: Status 404 returned error can't find the container with id f938555efed2357a865a44498d42192e81efdb27f0fd67b13502eb673e19caf5
Jul 02 23:15:36.351974 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:36.351934    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b" event={"ID":"cfff93ba-acef-4836-be23-4663650be31e","Type":"ContainerStarted","Data":"f938555efed2357a865a44498d42192e81efdb27f0fd67b13502eb673e19caf5"}
Jul 02 23:15:39.365012 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:39.364967    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b" event={"ID":"cfff93ba-acef-4836-be23-4663650be31e","Type":"ContainerStarted","Data":"1631aeca1725a71fc7e2f103e72003a8153c370b94cf0a0efaae5cdce0c2575a"}
Jul 02 23:15:40.370633 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:40.370596    2580 generic.go:356] "Generic (PLEG): container finished" podID="cfff93ba-acef-4836-be23-4663650be31e" containerID="1631aeca1725a71fc7e2f103e72003a8153c370b94cf0a0efaae5cdce0c2575a" exitCode=0
Jul 02 23:15:40.371087 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:40.370662    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b" event={"ID":"cfff93ba-acef-4836-be23-4663650be31e","Type":"ContainerDied","Data":"1631aeca1725a71fc7e2f103e72003a8153c370b94cf0a0efaae5cdce0c2575a"}
Jul 02 23:15:41.375394 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:41.375360    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b" event={"ID":"cfff93ba-acef-4836-be23-4663650be31e","Type":"ContainerStarted","Data":"fd37461f2bba7c72049732ec09bcb262f6f2a280e032066f63a73cdbdab7dd88"}
Jul 02 23:15:41.375775 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:41.375480    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:41.412314 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:41.412259    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b" podStartSLOduration=2.2458354959999998 podStartE2EDuration="6.412242847s" totalImagesPullingTime="3.126174453s" totalInitContainerRuntime="1.040232898s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:15:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:15:41.409972022 +0000 UTC m=+286.663762012" watchObservedRunningTime="2026-07-02 23:15:41.412242847 +0000 UTC m=+286.666032845"
Jul 02 23:15:52.381651 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:52.381618    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7d98bc9d58-svd4b"
Jul 02 23:15:55.170891 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.170862    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 02 23:15:55.475361 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.475326    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b76b84765-5x656"]
Jul 02 23:15:55.478362 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.478346    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b76b84765-5x656"
Jul 02 23:15:55.482547 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.482518    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 02 23:15:55.483706 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.483686    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jhstc\""
Jul 02 23:15:55.483892 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.483873    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 02 23:15:55.486921 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.486904    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 02 23:15:55.487085 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.487068    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 02 23:15:55.506058 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.506028    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b76b84765-5x656"]
Jul 02 23:15:55.618090 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.618055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0fa7374-6c60-44ba-8775-e5d1fc779b0d-tls-certs\") pod \"kube-auth-proxy-5b76b84765-5x656\" (UID: \"a0fa7374-6c60-44ba-8775-e5d1fc779b0d\") " pod="openshift-ingress/kube-auth-proxy-5b76b84765-5x656"
Jul 02 23:15:55.618276 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.618156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0fa7374-6c60-44ba-8775-e5d1fc779b0d-tmp\") pod \"kube-auth-proxy-5b76b84765-5x656\" (UID: \"a0fa7374-6c60-44ba-8775-e5d1fc779b0d\") " pod="openshift-ingress/kube-auth-proxy-5b76b84765-5x656"
Jul 02 23:15:55.618276 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.618205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29495\" (UniqueName: \"kubernetes.io/projected/a0fa7374-6c60-44ba-8775-e5d1fc779b0d-kube-api-access-29495\") pod \"kube-auth-proxy-5b76b84765-5x656\" (UID: \"a0fa7374-6c60-44ba-8775-e5d1fc779b0d\") " pod="openshift-ingress/kube-auth-proxy-5b76b84765-5x656"
Jul 02 23:15:55.687077 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.687043    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qbcpb"]
Jul 02 23:15:55.690005 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.689990    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qbcpb"
Jul 02 23:15:55.693177 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.693151    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-kb46z\""
Jul 02 23:15:55.694325 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.694310    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 02 23:15:55.712378 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.712348    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qbcpb"]
Jul 02 23:15:55.719323 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.719301    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29495\" (UniqueName: \"kubernetes.io/projected/a0fa7374-6c60-44ba-8775-e5d1fc779b0d-kube-api-access-29495\") pod \"kube-auth-proxy-5b76b84765-5x656\" (UID: \"a0fa7374-6c60-44ba-8775-e5d1fc779b0d\") " pod="openshift-ingress/kube-auth-proxy-5b76b84765-5x656"
Jul 02 23:15:55.719430 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.719358    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0fa7374-6c60-44ba-8775-e5d1fc779b0d-tls-certs\") pod \"kube-auth-proxy-5b76b84765-5x656\" (UID: \"a0fa7374-6c60-44ba-8775-e5d1fc779b0d\") " pod="openshift-ingress/kube-auth-proxy-5b76b84765-5x656"
Jul 02 23:15:55.719430 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.719422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0fa7374-6c60-44ba-8775-e5d1fc779b0d-tmp\") pod \"kube-auth-proxy-5b76b84765-5x656\" (UID: \"a0fa7374-6c60-44ba-8775-e5d1fc779b0d\") " pod="openshift-ingress/kube-auth-proxy-5b76b84765-5x656"
Jul 02 23:15:55.721998 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.721978    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0fa7374-6c60-44ba-8775-e5d1fc779b0d-tls-certs\") pod \"kube-auth-proxy-5b76b84765-5x656\" (UID: \"a0fa7374-6c60-44ba-8775-e5d1fc779b0d\") " pod="openshift-ingress/kube-auth-proxy-5b76b84765-5x656"
Jul 02 23:15:55.722067 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.722028    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0fa7374-6c60-44ba-8775-e5d1fc779b0d-tmp\") pod \"kube-auth-proxy-5b76b84765-5x656\" (UID: \"a0fa7374-6c60-44ba-8775-e5d1fc779b0d\") " pod="openshift-ingress/kube-auth-proxy-5b76b84765-5x656"
Jul 02 23:15:55.746418 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.746356    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29495\" (UniqueName: \"kubernetes.io/projected/a0fa7374-6c60-44ba-8775-e5d1fc779b0d-kube-api-access-29495\") pod \"kube-auth-proxy-5b76b84765-5x656\" (UID: \"a0fa7374-6c60-44ba-8775-e5d1fc779b0d\") " pod="openshift-ingress/kube-auth-proxy-5b76b84765-5x656"
Jul 02 23:15:55.788398 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.788371    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b76b84765-5x656"
Jul 02 23:15:55.820421 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.820393    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx6q5\" (UniqueName: \"kubernetes.io/projected/006fb957-9603-4c51-b3b4-dc012737ec89-kube-api-access-nx6q5\") pod \"odh-model-controller-7998c67dc4-qbcpb\" (UID: \"006fb957-9603-4c51-b3b4-dc012737ec89\") " pod="opendatahub/odh-model-controller-7998c67dc4-qbcpb"
Jul 02 23:15:55.820563 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.820453    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/006fb957-9603-4c51-b3b4-dc012737ec89-cert\") pod \"odh-model-controller-7998c67dc4-qbcpb\" (UID: \"006fb957-9603-4c51-b3b4-dc012737ec89\") " pod="opendatahub/odh-model-controller-7998c67dc4-qbcpb"
Jul 02 23:15:55.921485 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.921452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/006fb957-9603-4c51-b3b4-dc012737ec89-cert\") pod \"odh-model-controller-7998c67dc4-qbcpb\" (UID: \"006fb957-9603-4c51-b3b4-dc012737ec89\") " pod="opendatahub/odh-model-controller-7998c67dc4-qbcpb"
Jul 02 23:15:55.921676 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.921512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx6q5\" (UniqueName: \"kubernetes.io/projected/006fb957-9603-4c51-b3b4-dc012737ec89-kube-api-access-nx6q5\") pod \"odh-model-controller-7998c67dc4-qbcpb\" (UID: \"006fb957-9603-4c51-b3b4-dc012737ec89\") " pod="opendatahub/odh-model-controller-7998c67dc4-qbcpb"
Jul 02 23:15:55.923855 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.923830    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/006fb957-9603-4c51-b3b4-dc012737ec89-cert\") pod \"odh-model-controller-7998c67dc4-qbcpb\" (UID: \"006fb957-9603-4c51-b3b4-dc012737ec89\") " pod="opendatahub/odh-model-controller-7998c67dc4-qbcpb"
Jul 02 23:15:55.940160 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.940106    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx6q5\" (UniqueName: \"kubernetes.io/projected/006fb957-9603-4c51-b3b4-dc012737ec89-kube-api-access-nx6q5\") pod \"odh-model-controller-7998c67dc4-qbcpb\" (UID: \"006fb957-9603-4c51-b3b4-dc012737ec89\") " pod="opendatahub/odh-model-controller-7998c67dc4-qbcpb"
Jul 02 23:15:55.945465 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.945443    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b76b84765-5x656"]
Jul 02 23:15:55.949523 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:15:55.949498    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0fa7374_6c60_44ba_8775_e5d1fc779b0d.slice/crio-ee2abac6cfe568725c888ddde34f5890966752084ebb2e2c3c1867a16542a1d0 WatchSource:0}: Error finding container ee2abac6cfe568725c888ddde34f5890966752084ebb2e2c3c1867a16542a1d0: Status 404 returned error can't find the container with id ee2abac6cfe568725c888ddde34f5890966752084ebb2e2c3c1867a16542a1d0
Jul 02 23:15:55.999780 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:55.999721    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qbcpb"
Jul 02 23:15:56.183636 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:56.183601    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qbcpb"]
Jul 02 23:15:56.188651 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:15:56.188616    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod006fb957_9603_4c51_b3b4_dc012737ec89.slice/crio-7afd2d044bb83646175b494912c8874ccc2bb39b071b4da4131d1eb590159572 WatchSource:0}: Error finding container 7afd2d044bb83646175b494912c8874ccc2bb39b071b4da4131d1eb590159572: Status 404 returned error can't find the container with id 7afd2d044bb83646175b494912c8874ccc2bb39b071b4da4131d1eb590159572
Jul 02 23:15:56.429470 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:56.429377    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qbcpb" event={"ID":"006fb957-9603-4c51-b3b4-dc012737ec89","Type":"ContainerStarted","Data":"7afd2d044bb83646175b494912c8874ccc2bb39b071b4da4131d1eb590159572"}
Jul 02 23:15:56.430576 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:56.430544    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b76b84765-5x656" event={"ID":"a0fa7374-6c60-44ba-8775-e5d1fc779b0d","Type":"ContainerStarted","Data":"ee2abac6cfe568725c888ddde34f5890966752084ebb2e2c3c1867a16542a1d0"}
Jul 02 23:15:58.203032 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.202995    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"]
Jul 02 23:15:58.209989 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.209960    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.213819 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.213647    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 02 23:15:58.213819 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.213676    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 02 23:15:58.213819 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.213692    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 02 23:15:58.213819 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.213719    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 02 23:15:58.213819 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.213736    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jc486\""
Jul 02 23:15:58.229806 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.229776    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"]
Jul 02 23:15:58.343269 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.343232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.343454 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.343279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-cacerts\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.343454 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.343311    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5e9d50-7231-4c85-82d5-5d067476683c-local-certs\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.343454 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.343361    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.343454 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.343398    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.343454 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.343426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps9tm\" (UniqueName: \"kubernetes.io/projected/8c5e9d50-7231-4c85-82d5-5d067476683c-kube-api-access-ps9tm\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.343672 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.343467    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-token\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.444088 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.444053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.444296 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.444101    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-cacerts\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.444296 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.444151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5e9d50-7231-4c85-82d5-5d067476683c-local-certs\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.444296 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.444256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.444461 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.444301    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.444461 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.444329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps9tm\" (UniqueName: \"kubernetes.io/projected/8c5e9d50-7231-4c85-82d5-5d067476683c-kube-api-access-ps9tm\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.444461 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.444378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-token\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.445003 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.444949    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.446938 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.446866    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5e9d50-7231-4c85-82d5-5d067476683c-local-certs\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.447174 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.447105    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.447279 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.447186    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-cacerts\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.447279 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.447220    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.461766 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.461704    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps9tm\" (UniqueName: \"kubernetes.io/projected/8c5e9d50-7231-4c85-82d5-5d067476683c-kube-api-access-ps9tm\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.462812 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.462787    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-token\") pod \"istiod-openshift-gateway-7574459656-qxtd4\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:58.520819 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:58.520778    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:15:59.598862 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:59.598830    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"]
Jul 02 23:15:59.601779 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:15:59.601751    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c5e9d50_7231_4c85_82d5_5d067476683c.slice/crio-47a0a6b6b9ec3aa87f7c2c320a683680c9edd5eb8a228ec54fe0df8ff58058d9 WatchSource:0}: Error finding container 47a0a6b6b9ec3aa87f7c2c320a683680c9edd5eb8a228ec54fe0df8ff58058d9: Status 404 returned error can't find the container with id 47a0a6b6b9ec3aa87f7c2c320a683680c9edd5eb8a228ec54fe0df8ff58058d9
Jul 02 23:15:59.603717 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:15:59.603700    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 23:16:00.453387 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:00.453348    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qbcpb" event={"ID":"006fb957-9603-4c51-b3b4-dc012737ec89","Type":"ContainerStarted","Data":"08c1d480f92460bfc03fd2952ed7db7dc419576b4b562d83cf92b3d9acc4758e"}
Jul 02 23:16:00.454141 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:00.454094    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-qbcpb"
Jul 02 23:16:00.459543 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:00.459490    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b76b84765-5x656" event={"ID":"a0fa7374-6c60-44ba-8775-e5d1fc779b0d","Type":"ContainerStarted","Data":"9edf3cbec7e63f05b98a28924fa5ef72502f9be42f2de2794615d0b68b4a1ac0"}
Jul 02 23:16:00.461535 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:00.461504    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4" event={"ID":"8c5e9d50-7231-4c85-82d5-5d067476683c","Type":"ContainerStarted","Data":"47a0a6b6b9ec3aa87f7c2c320a683680c9edd5eb8a228ec54fe0df8ff58058d9"}
Jul 02 23:16:00.471059 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:00.471004    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-qbcpb" podStartSLOduration=2.199381345 podStartE2EDuration="5.470987144s" totalImagesPullingTime="3.271605799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:15:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:16:00.469619936 +0000 UTC m=+305.723409937" watchObservedRunningTime="2026-07-02 23:16:00.470987144 +0000 UTC m=+305.724777147"
Jul 02 23:16:00.495257 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:00.495205    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b76b84765-5x656" podStartSLOduration=1.97913765 podStartE2EDuration="5.495187063s" totalImagesPullingTime="3.516049413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:15:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:16:00.494317473 +0000 UTC m=+305.748107474" watchObservedRunningTime="2026-07-02 23:16:00.495187063 +0000 UTC m=+305.748977063"
Jul 02 23:16:02.213422 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:02.213371    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:16:02.213710 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:02.213442    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:16:02.472995 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:02.472896    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4" event={"ID":"8c5e9d50-7231-4c85-82d5-5d067476683c","Type":"ContainerStarted","Data":"a4f1c3cec08fc1baf7a2947b4b1ce9242298610e325847be816787c4d7858faf"}
Jul 02 23:16:02.473198 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:02.473049    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:16:02.501685 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:02.501625    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4" podStartSLOduration=1.8923148319999998 podStartE2EDuration="4.501609173s" totalImagesPullingTime="2.609294341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:15:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:16:02.500464915 +0000 UTC m=+307.754254937" watchObservedRunningTime="2026-07-02 23:16:02.501609173 +0000 UTC m=+307.755399171"
Jul 02 23:16:03.478672 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:03.478638    2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-qxtd4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 02 23:16:03.479040 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:03.478693    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4" podUID="8c5e9d50-7231-4c85-82d5-5d067476683c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 02 23:16:06.478252 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:06.478218    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:16:09.018566 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.018533    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr"]
Jul 02 23:16:09.025370 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.025349    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr"
Jul 02 23:16:09.029628 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.029611    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-g6zn4\""
Jul 02 23:16:09.029769 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.029615    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 02 23:16:09.089512 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.089486    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr"]
Jul 02 23:16:09.125767 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.125741    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a00b6be7-7525-4621-b309-07f433f73161-cert\") pod \"kserve-controller-manager-d6f4b65b8-w2pkr\" (UID: \"a00b6be7-7525-4621-b309-07f433f73161\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr"
Jul 02 23:16:09.125896 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.125799    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqk5n\" (UniqueName: \"kubernetes.io/projected/a00b6be7-7525-4621-b309-07f433f73161-kube-api-access-tqk5n\") pod \"kserve-controller-manager-d6f4b65b8-w2pkr\" (UID: \"a00b6be7-7525-4621-b309-07f433f73161\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr"
Jul 02 23:16:09.172182 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.172153    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9"]
Jul 02 23:16:09.181128 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.181103    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9"
Jul 02 23:16:09.190406 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.190386    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-bnd8c\""
Jul 02 23:16:09.190550 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.190533    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 02 23:16:09.207047 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.207027    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9"]
Jul 02 23:16:09.227008 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.226983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqk5n\" (UniqueName: \"kubernetes.io/projected/a00b6be7-7525-4621-b309-07f433f73161-kube-api-access-tqk5n\") pod \"kserve-controller-manager-d6f4b65b8-w2pkr\" (UID: \"a00b6be7-7525-4621-b309-07f433f73161\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr"
Jul 02 23:16:09.227152 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.227035    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvcdv\" (UniqueName: \"kubernetes.io/projected/3dbab3c8-c572-4831-b03e-1a94243d20b3-kube-api-access-tvcdv\") pod \"llmisvc-controller-manager-777c6bd7d5-5tvj9\" (UID: \"3dbab3c8-c572-4831-b03e-1a94243d20b3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9"
Jul 02 23:16:09.227198 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.227155    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3dbab3c8-c572-4831-b03e-1a94243d20b3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5tvj9\" (UID: \"3dbab3c8-c572-4831-b03e-1a94243d20b3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9"
Jul 02 23:16:09.227232 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.227199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a00b6be7-7525-4621-b309-07f433f73161-cert\") pod \"kserve-controller-manager-d6f4b65b8-w2pkr\" (UID: \"a00b6be7-7525-4621-b309-07f433f73161\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr"
Jul 02 23:16:09.229407 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.229384    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a00b6be7-7525-4621-b309-07f433f73161-cert\") pod \"kserve-controller-manager-d6f4b65b8-w2pkr\" (UID: \"a00b6be7-7525-4621-b309-07f433f73161\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr"
Jul 02 23:16:09.255306 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.255282    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqk5n\" (UniqueName: \"kubernetes.io/projected/a00b6be7-7525-4621-b309-07f433f73161-kube-api-access-tqk5n\") pod \"kserve-controller-manager-d6f4b65b8-w2pkr\" (UID: \"a00b6be7-7525-4621-b309-07f433f73161\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr"
Jul 02 23:16:09.327836 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.327756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvcdv\" (UniqueName: \"kubernetes.io/projected/3dbab3c8-c572-4831-b03e-1a94243d20b3-kube-api-access-tvcdv\") pod \"llmisvc-controller-manager-777c6bd7d5-5tvj9\" (UID: \"3dbab3c8-c572-4831-b03e-1a94243d20b3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9"
Jul 02 23:16:09.327836 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.327793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3dbab3c8-c572-4831-b03e-1a94243d20b3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5tvj9\" (UID: \"3dbab3c8-c572-4831-b03e-1a94243d20b3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9"
Jul 02 23:16:09.330100 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.330076    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3dbab3c8-c572-4831-b03e-1a94243d20b3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5tvj9\" (UID: \"3dbab3c8-c572-4831-b03e-1a94243d20b3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9"
Jul 02 23:16:09.334981 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.334960    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr"
Jul 02 23:16:09.346921 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.346897    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvcdv\" (UniqueName: \"kubernetes.io/projected/3dbab3c8-c572-4831-b03e-1a94243d20b3-kube-api-access-tvcdv\") pod \"llmisvc-controller-manager-777c6bd7d5-5tvj9\" (UID: \"3dbab3c8-c572-4831-b03e-1a94243d20b3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9"
Jul 02 23:16:09.490222 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.490196    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9"
Jul 02 23:16:09.515986 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.515902    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr"]
Jul 02 23:16:09.518470 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:16:09.518444    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda00b6be7_7525_4621_b309_07f433f73161.slice/crio-ae35a24c61b7bea33ec4ac9c396b8075529d498bce2fbd6403b2685f9457d13b WatchSource:0}: Error finding container ae35a24c61b7bea33ec4ac9c396b8075529d498bce2fbd6403b2685f9457d13b: Status 404 returned error can't find the container with id ae35a24c61b7bea33ec4ac9c396b8075529d498bce2fbd6403b2685f9457d13b
Jul 02 23:16:09.685710 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:09.685673    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9"]
Jul 02 23:16:09.691527 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:16:09.691501    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3dbab3c8_c572_4831_b03e_1a94243d20b3.slice/crio-340cc8669ff85d9648d8828ad4173ecd4e3d30a0d45e9090953ef622c8caff27 WatchSource:0}: Error finding container 340cc8669ff85d9648d8828ad4173ecd4e3d30a0d45e9090953ef622c8caff27: Status 404 returned error can't find the container with id 340cc8669ff85d9648d8828ad4173ecd4e3d30a0d45e9090953ef622c8caff27
Jul 02 23:16:10.509927 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:10.509837    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr" event={"ID":"a00b6be7-7525-4621-b309-07f433f73161","Type":"ContainerStarted","Data":"ae35a24c61b7bea33ec4ac9c396b8075529d498bce2fbd6403b2685f9457d13b"}
Jul 02 23:16:10.511983 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:10.511945    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9" event={"ID":"3dbab3c8-c572-4831-b03e-1a94243d20b3","Type":"ContainerStarted","Data":"340cc8669ff85d9648d8828ad4173ecd4e3d30a0d45e9090953ef622c8caff27"}
Jul 02 23:16:12.475917 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:12.475881    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-qbcpb"
Jul 02 23:16:12.522300 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:12.522259    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9" event={"ID":"3dbab3c8-c572-4831-b03e-1a94243d20b3","Type":"ContainerStarted","Data":"80a5e60966f00b122dbd6a8d840a0b013ea877255ad8056f6188d0240c5d57c5"}
Jul 02 23:16:12.522479 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:12.522420    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9"
Jul 02 23:16:12.524024 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:12.523959    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr" event={"ID":"a00b6be7-7525-4621-b309-07f433f73161","Type":"ContainerStarted","Data":"eb4e4beb7bf2ecc5c87b74db93d7be19c52647c013a7dcc46023889015083c9e"}
Jul 02 23:16:12.524291 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:12.524267    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr"
Jul 02 23:16:12.553412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:12.553366    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9" podStartSLOduration=1.041025276 podStartE2EDuration="3.553349946s" totalImagesPullingTime="2.51232467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:16:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:16:12.55105065 +0000 UTC m=+317.804840648" watchObservedRunningTime="2026-07-02 23:16:12.553349946 +0000 UTC m=+317.807139948"
Jul 02 23:16:12.576784 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:12.576735    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr" podStartSLOduration=2.539911712 podStartE2EDuration="4.576721531s" totalImagesPullingTime="2.036809819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:16:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:16:12.575382713 +0000 UTC m=+317.829172712" watchObservedRunningTime="2026-07-02 23:16:12.576721531 +0000 UTC m=+317.830511528"
Jul 02 23:16:31.622852 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:31.622824    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"]
Jul 02 23:16:31.623296 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:31.623038    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4" podUID="8c5e9d50-7231-4c85-82d5-5d067476683c" containerName="discovery" containerID="cri-o://a4f1c3cec08fc1baf7a2947b4b1ce9242298610e325847be816787c4d7858faf" gracePeriod=30
Jul 02 23:16:31.896624 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:31.896551    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:16:32.014231 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.014195    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-csr-dns-cert\") pod \"8c5e9d50-7231-4c85-82d5-5d067476683c\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") "
Jul 02 23:16:32.014393 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.014248    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5e9d50-7231-4c85-82d5-5d067476683c-local-certs\") pod \"8c5e9d50-7231-4c85-82d5-5d067476683c\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") "
Jul 02 23:16:32.014393 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.014269    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-cacerts\") pod \"8c5e9d50-7231-4c85-82d5-5d067476683c\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") "
Jul 02 23:16:32.014393 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.014372    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-token\") pod \"8c5e9d50-7231-4c85-82d5-5d067476683c\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") "
Jul 02 23:16:32.014530 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.014417    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-kubeconfig\") pod \"8c5e9d50-7231-4c85-82d5-5d067476683c\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") "
Jul 02 23:16:32.014530 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.014453    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-csr-ca-configmap\") pod \"8c5e9d50-7231-4c85-82d5-5d067476683c\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") "
Jul 02 23:16:32.014530 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.014483    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ps9tm\" (UniqueName: \"kubernetes.io/projected/8c5e9d50-7231-4c85-82d5-5d067476683c-kube-api-access-ps9tm\") pod \"8c5e9d50-7231-4c85-82d5-5d067476683c\" (UID: \"8c5e9d50-7231-4c85-82d5-5d067476683c\") "
Jul 02 23:16:32.014861 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.014827    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "8c5e9d50-7231-4c85-82d5-5d067476683c" (UID: "8c5e9d50-7231-4c85-82d5-5d067476683c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 23:16:32.016646 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.016613    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-token" (OuterVolumeSpecName: "istio-token") pod "8c5e9d50-7231-4c85-82d5-5d067476683c" (UID: "8c5e9d50-7231-4c85-82d5-5d067476683c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:16:32.016747 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.016620    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-cacerts" (OuterVolumeSpecName: "cacerts") pod "8c5e9d50-7231-4c85-82d5-5d067476683c" (UID: "8c5e9d50-7231-4c85-82d5-5d067476683c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 23:16:32.016807 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.016781    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "8c5e9d50-7231-4c85-82d5-5d067476683c" (UID: "8c5e9d50-7231-4c85-82d5-5d067476683c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 23:16:32.017024 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.017000    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "8c5e9d50-7231-4c85-82d5-5d067476683c" (UID: "8c5e9d50-7231-4c85-82d5-5d067476683c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 23:16:32.017089 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.017037    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c5e9d50-7231-4c85-82d5-5d067476683c-kube-api-access-ps9tm" (OuterVolumeSpecName: "kube-api-access-ps9tm") pod "8c5e9d50-7231-4c85-82d5-5d067476683c" (UID: "8c5e9d50-7231-4c85-82d5-5d067476683c"). InnerVolumeSpecName "kube-api-access-ps9tm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:16:32.017089 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.017045    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c5e9d50-7231-4c85-82d5-5d067476683c-local-certs" (OuterVolumeSpecName: "local-certs") pod "8c5e9d50-7231-4c85-82d5-5d067476683c" (UID: "8c5e9d50-7231-4c85-82d5-5d067476683c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:16:32.115487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.115454    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-csr-dns-cert\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:16:32.115487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.115481    2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5e9d50-7231-4c85-82d5-5d067476683c-local-certs\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:16:32.115487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.115491    2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-cacerts\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:16:32.115487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.115498    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-token\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:16:32.115736 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.115505    2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-kubeconfig\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:16:32.115736 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.115513    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8c5e9d50-7231-4c85-82d5-5d067476683c-istio-csr-ca-configmap\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:16:32.115736 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.115521    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ps9tm\" (UniqueName: \"kubernetes.io/projected/8c5e9d50-7231-4c85-82d5-5d067476683c-kube-api-access-ps9tm\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:16:32.601531 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.601493    2580 generic.go:356] "Generic (PLEG): container finished" podID="8c5e9d50-7231-4c85-82d5-5d067476683c" containerID="a4f1c3cec08fc1baf7a2947b4b1ce9242298610e325847be816787c4d7858faf" exitCode=0
Jul 02 23:16:32.601754 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.601541    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"
Jul 02 23:16:32.601754 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.601571    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4" event={"ID":"8c5e9d50-7231-4c85-82d5-5d067476683c","Type":"ContainerDied","Data":"a4f1c3cec08fc1baf7a2947b4b1ce9242298610e325847be816787c4d7858faf"}
Jul 02 23:16:32.601754 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.601621    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4" event={"ID":"8c5e9d50-7231-4c85-82d5-5d067476683c","Type":"ContainerDied","Data":"47a0a6b6b9ec3aa87f7c2c320a683680c9edd5eb8a228ec54fe0df8ff58058d9"}
Jul 02 23:16:32.601754 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.601640    2580 scope.go:117] "RemoveContainer" containerID="a4f1c3cec08fc1baf7a2947b4b1ce9242298610e325847be816787c4d7858faf"
Jul 02 23:16:32.609712 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.609696    2580 scope.go:117] "RemoveContainer" containerID="a4f1c3cec08fc1baf7a2947b4b1ce9242298610e325847be816787c4d7858faf"
Jul 02 23:16:32.609988 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:16:32.609971    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4f1c3cec08fc1baf7a2947b4b1ce9242298610e325847be816787c4d7858faf\": container with ID starting with a4f1c3cec08fc1baf7a2947b4b1ce9242298610e325847be816787c4d7858faf not found: ID does not exist" containerID="a4f1c3cec08fc1baf7a2947b4b1ce9242298610e325847be816787c4d7858faf"
Jul 02 23:16:32.610046 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.609998    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f1c3cec08fc1baf7a2947b4b1ce9242298610e325847be816787c4d7858faf"} err="failed to get container status \"a4f1c3cec08fc1baf7a2947b4b1ce9242298610e325847be816787c4d7858faf\": rpc error: code = NotFound desc = could not find container \"a4f1c3cec08fc1baf7a2947b4b1ce9242298610e325847be816787c4d7858faf\": container with ID starting with a4f1c3cec08fc1baf7a2947b4b1ce9242298610e325847be816787c4d7858faf not found: ID does not exist"
Jul 02 23:16:32.624164 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.624139    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"]
Jul 02 23:16:32.628335 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:32.628315    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qxtd4"]
Jul 02 23:16:33.281771 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:33.281736    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c5e9d50-7231-4c85-82d5-5d067476683c" path="/var/lib/kubelet/pods/8c5e9d50-7231-4c85-82d5-5d067476683c/volumes"
Jul 02 23:16:43.531430 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:43.531395    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5tvj9"
Jul 02 23:16:43.534110 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:43.534089    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w2pkr"
Jul 02 23:16:56.372823 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.372790    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s4m7t"]
Jul 02 23:16:56.373235 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.373074    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c5e9d50-7231-4c85-82d5-5d067476683c" containerName="discovery"
Jul 02 23:16:56.373235 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.373085    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c5e9d50-7231-4c85-82d5-5d067476683c" containerName="discovery"
Jul 02 23:16:56.373235 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.373167    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c5e9d50-7231-4c85-82d5-5d067476683c" containerName="discovery"
Jul 02 23:16:56.376883 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.376861    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-s4m7t"
Jul 02 23:16:56.381759 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.381734    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 02 23:16:56.382607 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.382584    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 02 23:16:56.382700 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.382587    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-qqhjf\""
Jul 02 23:16:56.390375 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.390355    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s4m7t"]
Jul 02 23:16:56.488509 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.488473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68gh5\" (UniqueName: \"kubernetes.io/projected/192768a2-a24b-40bb-968e-457ef1be597e-kube-api-access-68gh5\") pod \"kuadrant-operator-catalog-s4m7t\" (UID: \"192768a2-a24b-40bb-968e-457ef1be597e\") " pod="kuadrant-system/kuadrant-operator-catalog-s4m7t"
Jul 02 23:16:56.588807 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.588775    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68gh5\" (UniqueName: \"kubernetes.io/projected/192768a2-a24b-40bb-968e-457ef1be597e-kube-api-access-68gh5\") pod \"kuadrant-operator-catalog-s4m7t\" (UID: \"192768a2-a24b-40bb-968e-457ef1be597e\") " pod="kuadrant-system/kuadrant-operator-catalog-s4m7t"
Jul 02 23:16:56.597739 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.597716    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68gh5\" (UniqueName: \"kubernetes.io/projected/192768a2-a24b-40bb-968e-457ef1be597e-kube-api-access-68gh5\") pod \"kuadrant-operator-catalog-s4m7t\" (UID: \"192768a2-a24b-40bb-968e-457ef1be597e\") " pod="kuadrant-system/kuadrant-operator-catalog-s4m7t"
Jul 02 23:16:56.688590 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.688556    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-s4m7t"
Jul 02 23:16:56.721660 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.721617    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s4m7t"]
Jul 02 23:16:56.816227 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.816033    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s4m7t"]
Jul 02 23:16:56.818843 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:16:56.818815    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod192768a2_a24b_40bb_968e_457ef1be597e.slice/crio-38f16902b51d34fcd92f31dc4ce29e9ac41a575da7a4ef2a6ed56221f972e9db WatchSource:0}: Error finding container 38f16902b51d34fcd92f31dc4ce29e9ac41a575da7a4ef2a6ed56221f972e9db: Status 404 returned error can't find the container with id 38f16902b51d34fcd92f31dc4ce29e9ac41a575da7a4ef2a6ed56221f972e9db
Jul 02 23:16:56.933099 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.933062    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zj7mb"]
Jul 02 23:16:56.937428 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.937403    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zj7mb"
Jul 02 23:16:56.946869 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:56.946843    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zj7mb"]
Jul 02 23:16:57.093555 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:57.093527    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w7v6\" (UniqueName: \"kubernetes.io/projected/ada81472-d26c-4ef8-b31d-4ebdf4dd9132-kube-api-access-9w7v6\") pod \"kuadrant-operator-catalog-zj7mb\" (UID: \"ada81472-d26c-4ef8-b31d-4ebdf4dd9132\") " pod="kuadrant-system/kuadrant-operator-catalog-zj7mb"
Jul 02 23:16:57.194647 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:57.194575    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w7v6\" (UniqueName: \"kubernetes.io/projected/ada81472-d26c-4ef8-b31d-4ebdf4dd9132-kube-api-access-9w7v6\") pod \"kuadrant-operator-catalog-zj7mb\" (UID: \"ada81472-d26c-4ef8-b31d-4ebdf4dd9132\") " pod="kuadrant-system/kuadrant-operator-catalog-zj7mb"
Jul 02 23:16:57.203401 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:57.203375    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w7v6\" (UniqueName: \"kubernetes.io/projected/ada81472-d26c-4ef8-b31d-4ebdf4dd9132-kube-api-access-9w7v6\") pod \"kuadrant-operator-catalog-zj7mb\" (UID: \"ada81472-d26c-4ef8-b31d-4ebdf4dd9132\") " pod="kuadrant-system/kuadrant-operator-catalog-zj7mb"
Jul 02 23:16:57.246567 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:57.246534    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zj7mb"
Jul 02 23:16:57.423807 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:57.423546    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zj7mb"]
Jul 02 23:16:57.459822 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:16:57.459746    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada81472_d26c_4ef8_b31d_4ebdf4dd9132.slice/crio-fce8177fbdeca71f42075f5a7bd71acd9490a306d80a304c5e69be34a659b7cb WatchSource:0}: Error finding container fce8177fbdeca71f42075f5a7bd71acd9490a306d80a304c5e69be34a659b7cb: Status 404 returned error can't find the container with id fce8177fbdeca71f42075f5a7bd71acd9490a306d80a304c5e69be34a659b7cb
Jul 02 23:16:57.700130 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:57.700070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zj7mb" event={"ID":"ada81472-d26c-4ef8-b31d-4ebdf4dd9132","Type":"ContainerStarted","Data":"fce8177fbdeca71f42075f5a7bd71acd9490a306d80a304c5e69be34a659b7cb"}
Jul 02 23:16:57.701649 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:57.701619    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-s4m7t" event={"ID":"192768a2-a24b-40bb-968e-457ef1be597e","Type":"ContainerStarted","Data":"38f16902b51d34fcd92f31dc4ce29e9ac41a575da7a4ef2a6ed56221f972e9db"}
Jul 02 23:16:58.706627 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:58.706575    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zj7mb" event={"ID":"ada81472-d26c-4ef8-b31d-4ebdf4dd9132","Type":"ContainerStarted","Data":"bcd0586b8f778030d227fe844e81ed3a8020d96cb8f0bc4f3898a37656a0fb3c"}
Jul 02 23:16:58.708341 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:58.708310    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-s4m7t" event={"ID":"192768a2-a24b-40bb-968e-457ef1be597e","Type":"ContainerStarted","Data":"b4a6406eec2857468be84c62d480afd5231a1fde4c276ea92bb185275cb1e8e1"}
Jul 02 23:16:58.708451 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:58.708394    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-s4m7t" podUID="192768a2-a24b-40bb-968e-457ef1be597e" containerName="registry-server" containerID="cri-o://b4a6406eec2857468be84c62d480afd5231a1fde4c276ea92bb185275cb1e8e1" gracePeriod=2
Jul 02 23:16:58.726440 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:58.726392    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-zj7mb" podStartSLOduration=1.583274069 podStartE2EDuration="2.726375131s" totalImagesPullingTime="1.143101062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:16:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:16:58.724792985 +0000 UTC m=+363.978582984" watchObservedRunningTime="2026-07-02 23:16:58.726375131 +0000 UTC m=+363.980165136"
Jul 02 23:16:58.742884 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:58.742824    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-s4m7t" podStartSLOduration=0.959233292 podStartE2EDuration="2.742802975s" totalImagesPullingTime="1.783569683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:16:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:16:58.742001643 +0000 UTC m=+363.995791642" watchObservedRunningTime="2026-07-02 23:16:58.742802975 +0000 UTC m=+363.996592974"
Jul 02 23:16:58.941613 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:58.941592    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-s4m7t_192768a2-a24b-40bb-968e-457ef1be597e/registry-server/0.log"
Jul 02 23:16:58.943937 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:58.943920    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-s4m7t"
Jul 02 23:16:59.010775 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:59.010738    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68gh5\" (UniqueName: \"kubernetes.io/projected/192768a2-a24b-40bb-968e-457ef1be597e-kube-api-access-68gh5\") pod \"192768a2-a24b-40bb-968e-457ef1be597e\" (UID: \"192768a2-a24b-40bb-968e-457ef1be597e\") "
Jul 02 23:16:59.012799 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:59.012777    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/192768a2-a24b-40bb-968e-457ef1be597e-kube-api-access-68gh5" (OuterVolumeSpecName: "kube-api-access-68gh5") pod "192768a2-a24b-40bb-968e-457ef1be597e" (UID: "192768a2-a24b-40bb-968e-457ef1be597e"). InnerVolumeSpecName "kube-api-access-68gh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:16:59.111437 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:59.111408    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68gh5\" (UniqueName: \"kubernetes.io/projected/192768a2-a24b-40bb-968e-457ef1be597e-kube-api-access-68gh5\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:16:59.713580 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:59.713551    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-s4m7t_192768a2-a24b-40bb-968e-457ef1be597e/registry-server/0.log"
Jul 02 23:16:59.714026 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:59.713593    2580 generic.go:356] "Generic (PLEG): container finished" podID="192768a2-a24b-40bb-968e-457ef1be597e" containerID="b4a6406eec2857468be84c62d480afd5231a1fde4c276ea92bb185275cb1e8e1" exitCode=1
Jul 02 23:16:59.714026 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:59.713633    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-s4m7t"
Jul 02 23:16:59.714026 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:59.713674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-s4m7t" event={"ID":"192768a2-a24b-40bb-968e-457ef1be597e","Type":"ContainerDied","Data":"b4a6406eec2857468be84c62d480afd5231a1fde4c276ea92bb185275cb1e8e1"}
Jul 02 23:16:59.714026 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:59.713725    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-s4m7t" event={"ID":"192768a2-a24b-40bb-968e-457ef1be597e","Type":"ContainerDied","Data":"38f16902b51d34fcd92f31dc4ce29e9ac41a575da7a4ef2a6ed56221f972e9db"}
Jul 02 23:16:59.714026 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:59.713750    2580 scope.go:117] "RemoveContainer" containerID="b4a6406eec2857468be84c62d480afd5231a1fde4c276ea92bb185275cb1e8e1"
Jul 02 23:16:59.722157 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:59.722135    2580 scope.go:117] "RemoveContainer" containerID="b4a6406eec2857468be84c62d480afd5231a1fde4c276ea92bb185275cb1e8e1"
Jul 02 23:16:59.722406 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:16:59.722389    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4a6406eec2857468be84c62d480afd5231a1fde4c276ea92bb185275cb1e8e1\": container with ID starting with b4a6406eec2857468be84c62d480afd5231a1fde4c276ea92bb185275cb1e8e1 not found: ID does not exist" containerID="b4a6406eec2857468be84c62d480afd5231a1fde4c276ea92bb185275cb1e8e1"
Jul 02 23:16:59.722471 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:59.722416    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4a6406eec2857468be84c62d480afd5231a1fde4c276ea92bb185275cb1e8e1"} err="failed to get container status \"b4a6406eec2857468be84c62d480afd5231a1fde4c276ea92bb185275cb1e8e1\": rpc error: code = NotFound desc = could not find container \"b4a6406eec2857468be84c62d480afd5231a1fde4c276ea92bb185275cb1e8e1\": container with ID starting with b4a6406eec2857468be84c62d480afd5231a1fde4c276ea92bb185275cb1e8e1 not found: ID does not exist"
Jul 02 23:16:59.730266 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:59.730247    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s4m7t"]
Jul 02 23:16:59.734278 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:16:59.734256    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s4m7t"]
Jul 02 23:17:01.286440 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:01.286403    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="192768a2-a24b-40bb-968e-457ef1be597e" path="/var/lib/kubelet/pods/192768a2-a24b-40bb-968e-457ef1be597e/volumes"
Jul 02 23:17:07.247007 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:07.246947    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-zj7mb"
Jul 02 23:17:07.247542 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:07.247029    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-zj7mb"
Jul 02 23:17:07.268675 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:07.268648    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-zj7mb"
Jul 02 23:17:07.767041 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:07.767016    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-zj7mb"
Jul 02 23:17:11.417840 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.417804    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"]
Jul 02 23:17:11.418236 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.418111    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="192768a2-a24b-40bb-968e-457ef1be597e" containerName="registry-server"
Jul 02 23:17:11.418236 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.418138    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="192768a2-a24b-40bb-968e-457ef1be597e" containerName="registry-server"
Jul 02 23:17:11.418236 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.418204    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="192768a2-a24b-40bb-968e-457ef1be597e" containerName="registry-server"
Jul 02 23:17:11.422661 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.422641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"
Jul 02 23:17:11.427652 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.427630    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-22wxt\""
Jul 02 23:17:11.434136 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.434094    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"]
Jul 02 23:17:11.492348 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.492315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/91527594-f1c6-4f49-bbd0-f862aa1d3477-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc\" (UID: \"91527594-f1c6-4f49-bbd0-f862aa1d3477\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"
Jul 02 23:17:11.492515 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.492361    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/91527594-f1c6-4f49-bbd0-f862aa1d3477-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc\" (UID: \"91527594-f1c6-4f49-bbd0-f862aa1d3477\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"
Jul 02 23:17:11.492515 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.492391    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dlb9\" (UniqueName: \"kubernetes.io/projected/91527594-f1c6-4f49-bbd0-f862aa1d3477-kube-api-access-6dlb9\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc\" (UID: \"91527594-f1c6-4f49-bbd0-f862aa1d3477\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"
Jul 02 23:17:11.593239 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.593206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/91527594-f1c6-4f49-bbd0-f862aa1d3477-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc\" (UID: \"91527594-f1c6-4f49-bbd0-f862aa1d3477\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"
Jul 02 23:17:11.593406 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.593248    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dlb9\" (UniqueName: \"kubernetes.io/projected/91527594-f1c6-4f49-bbd0-f862aa1d3477-kube-api-access-6dlb9\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc\" (UID: \"91527594-f1c6-4f49-bbd0-f862aa1d3477\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"
Jul 02 23:17:11.593406 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.593282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/91527594-f1c6-4f49-bbd0-f862aa1d3477-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc\" (UID: \"91527594-f1c6-4f49-bbd0-f862aa1d3477\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"
Jul 02 23:17:11.593621 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.593598    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/91527594-f1c6-4f49-bbd0-f862aa1d3477-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc\" (UID: \"91527594-f1c6-4f49-bbd0-f862aa1d3477\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"
Jul 02 23:17:11.593656 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.593607    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/91527594-f1c6-4f49-bbd0-f862aa1d3477-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc\" (UID: \"91527594-f1c6-4f49-bbd0-f862aa1d3477\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"
Jul 02 23:17:11.602629 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.602600    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dlb9\" (UniqueName: \"kubernetes.io/projected/91527594-f1c6-4f49-bbd0-f862aa1d3477-kube-api-access-6dlb9\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc\" (UID: \"91527594-f1c6-4f49-bbd0-f862aa1d3477\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"
Jul 02 23:17:11.730959 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.730922    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"
Jul 02 23:17:11.870383 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:11.870356    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"]
Jul 02 23:17:11.872859 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:17:11.872825    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91527594_f1c6_4f49_bbd0_f862aa1d3477.slice/crio-9439bd5c6b659456a221d1707670f3b2ea9075890a4c21899536c4b1bc78fa9d WatchSource:0}: Error finding container 9439bd5c6b659456a221d1707670f3b2ea9075890a4c21899536c4b1bc78fa9d: Status 404 returned error can't find the container with id 9439bd5c6b659456a221d1707670f3b2ea9075890a4c21899536c4b1bc78fa9d
Jul 02 23:17:12.150207 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.150074    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"]
Jul 02 23:17:12.153344 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.153320    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"
Jul 02 23:17:12.161996 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.161968    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"]
Jul 02 23:17:12.197866 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.197833    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6lxm\" (UniqueName: \"kubernetes.io/projected/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-kube-api-access-g6lxm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn\" (UID: \"2bbc37b4-d84e-44b5-a3f3-86cb542915b0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"
Jul 02 23:17:12.198034 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.197918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn\" (UID: \"2bbc37b4-d84e-44b5-a3f3-86cb542915b0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"
Jul 02 23:17:12.198034 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.197939    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn\" (UID: \"2bbc37b4-d84e-44b5-a3f3-86cb542915b0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"
Jul 02 23:17:12.299233 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.299197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6lxm\" (UniqueName: \"kubernetes.io/projected/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-kube-api-access-g6lxm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn\" (UID: \"2bbc37b4-d84e-44b5-a3f3-86cb542915b0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"
Jul 02 23:17:12.299407 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.299298    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn\" (UID: \"2bbc37b4-d84e-44b5-a3f3-86cb542915b0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"
Jul 02 23:17:12.299407 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.299318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn\" (UID: \"2bbc37b4-d84e-44b5-a3f3-86cb542915b0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"
Jul 02 23:17:12.299727 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.299705    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn\" (UID: \"2bbc37b4-d84e-44b5-a3f3-86cb542915b0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"
Jul 02 23:17:12.299786 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.299772    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn\" (UID: \"2bbc37b4-d84e-44b5-a3f3-86cb542915b0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"
Jul 02 23:17:12.307786 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.307757    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6lxm\" (UniqueName: \"kubernetes.io/projected/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-kube-api-access-g6lxm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn\" (UID: \"2bbc37b4-d84e-44b5-a3f3-86cb542915b0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"
Jul 02 23:17:12.462535 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.462504    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"
Jul 02 23:17:12.587809 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.587785    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"]
Jul 02 23:17:12.589987 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:17:12.589962    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bbc37b4_d84e_44b5_a3f3_86cb542915b0.slice/crio-4018b3953cfbd7eb67a821bc204454ff8ed6814e75b256afc65c38d8063879f9 WatchSource:0}: Error finding container 4018b3953cfbd7eb67a821bc204454ff8ed6814e75b256afc65c38d8063879f9: Status 404 returned error can't find the container with id 4018b3953cfbd7eb67a821bc204454ff8ed6814e75b256afc65c38d8063879f9
Jul 02 23:17:12.769632 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.769547    2580 generic.go:356] "Generic (PLEG): container finished" podID="2bbc37b4-d84e-44b5-a3f3-86cb542915b0" containerID="d1759ef5c212296e5238907670300f7b9c1d266d6c0bd9a8df4225d6a921ff3e" exitCode=0
Jul 02 23:17:12.769632 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.769615    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn" event={"ID":"2bbc37b4-d84e-44b5-a3f3-86cb542915b0","Type":"ContainerDied","Data":"d1759ef5c212296e5238907670300f7b9c1d266d6c0bd9a8df4225d6a921ff3e"}
Jul 02 23:17:12.769854 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.769651    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn" event={"ID":"2bbc37b4-d84e-44b5-a3f3-86cb542915b0","Type":"ContainerStarted","Data":"4018b3953cfbd7eb67a821bc204454ff8ed6814e75b256afc65c38d8063879f9"}
Jul 02 23:17:12.771322 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.771298    2580 generic.go:356] "Generic (PLEG): container finished" podID="91527594-f1c6-4f49-bbd0-f862aa1d3477" containerID="a6f355bba6a3fe339c80d5db0bcb17081ee61bf361b49f8797648a43615fd7ac" exitCode=0
Jul 02 23:17:12.771419 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.771333    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc" event={"ID":"91527594-f1c6-4f49-bbd0-f862aa1d3477","Type":"ContainerDied","Data":"a6f355bba6a3fe339c80d5db0bcb17081ee61bf361b49f8797648a43615fd7ac"}
Jul 02 23:17:12.771419 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.771375    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc" event={"ID":"91527594-f1c6-4f49-bbd0-f862aa1d3477","Type":"ContainerStarted","Data":"9439bd5c6b659456a221d1707670f3b2ea9075890a4c21899536c4b1bc78fa9d"}
Jul 02 23:17:12.953979 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.953944    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"]
Jul 02 23:17:12.957306 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.957289    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"
Jul 02 23:17:12.972755 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:12.972723    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"]
Jul 02 23:17:13.005525 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.005492    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb9cp\" (UniqueName: \"kubernetes.io/projected/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-kube-api-access-cb9cp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g\" (UID: \"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"
Jul 02 23:17:13.005525 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.005527    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g\" (UID: \"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"
Jul 02 23:17:13.005719 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.005567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g\" (UID: \"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"
Jul 02 23:17:13.106015 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.105925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb9cp\" (UniqueName: \"kubernetes.io/projected/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-kube-api-access-cb9cp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g\" (UID: \"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"
Jul 02 23:17:13.106015 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.105971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g\" (UID: \"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"
Jul 02 23:17:13.106015 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.106008    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g\" (UID: \"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"
Jul 02 23:17:13.106384 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.106368    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g\" (UID: \"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"
Jul 02 23:17:13.106424 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.106398    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g\" (UID: \"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"
Jul 02 23:17:13.115825 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.115795    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb9cp\" (UniqueName: \"kubernetes.io/projected/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-kube-api-access-cb9cp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g\" (UID: \"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"
Jul 02 23:17:13.265628 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.265602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"
Jul 02 23:17:13.561872 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.561844    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"]
Jul 02 23:17:13.595077 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:17:13.595046    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b8fc35a_4cf3_4c5d_b5b1_718a91be02eb.slice/crio-aad649fe0475bef069277eea0e98fd741a5a704471ccd67e24445e865b511e37 WatchSource:0}: Error finding container aad649fe0475bef069277eea0e98fd741a5a704471ccd67e24445e865b511e37: Status 404 returned error can't find the container with id aad649fe0475bef069277eea0e98fd741a5a704471ccd67e24445e865b511e37
Jul 02 23:17:13.744292 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.744259    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"]
Jul 02 23:17:13.747694 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.747677    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"
Jul 02 23:17:13.761140 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.758585    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"]
Jul 02 23:17:13.777280 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.777248    2580 generic.go:356] "Generic (PLEG): container finished" podID="0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb" containerID="3b2046014cdb8020b49ce3eb889eda4c8261a6117b7747a6fc917b76a7ce6652" exitCode=0
Jul 02 23:17:13.777415 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.777323    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g" event={"ID":"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb","Type":"ContainerDied","Data":"3b2046014cdb8020b49ce3eb889eda4c8261a6117b7747a6fc917b76a7ce6652"}
Jul 02 23:17:13.777415 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.777369    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g" event={"ID":"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb","Type":"ContainerStarted","Data":"aad649fe0475bef069277eea0e98fd741a5a704471ccd67e24445e865b511e37"}
Jul 02 23:17:13.779264 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.779239    2580 generic.go:356] "Generic (PLEG): container finished" podID="2bbc37b4-d84e-44b5-a3f3-86cb542915b0" containerID="ae4ebf62370824f51c3844ab4ff37615a639c74f215d76d04dc2acc6753dc01c" exitCode=0
Jul 02 23:17:13.779390 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.779317    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn" event={"ID":"2bbc37b4-d84e-44b5-a3f3-86cb542915b0","Type":"ContainerDied","Data":"ae4ebf62370824f51c3844ab4ff37615a639c74f215d76d04dc2acc6753dc01c"}
Jul 02 23:17:13.781231 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.781209    2580 generic.go:356] "Generic (PLEG): container finished" podID="91527594-f1c6-4f49-bbd0-f862aa1d3477" containerID="5e4dceb8ac14bdbb6491fa8254ac22a39cdf050af70e4e7a1d4984602b54dc4f" exitCode=0
Jul 02 23:17:13.781330 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.781312    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc" event={"ID":"91527594-f1c6-4f49-bbd0-f862aa1d3477","Type":"ContainerDied","Data":"5e4dceb8ac14bdbb6491fa8254ac22a39cdf050af70e4e7a1d4984602b54dc4f"}
Jul 02 23:17:13.812630 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.812598    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0b9cf75-f964-4467-ace5-5362020b4be7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb\" (UID: \"c0b9cf75-f964-4467-ace5-5362020b4be7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"
Jul 02 23:17:13.812735 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.812644    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cskn4\" (UniqueName: \"kubernetes.io/projected/c0b9cf75-f964-4467-ace5-5362020b4be7-kube-api-access-cskn4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb\" (UID: \"c0b9cf75-f964-4467-ace5-5362020b4be7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"
Jul 02 23:17:13.812776 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.812739    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0b9cf75-f964-4467-ace5-5362020b4be7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb\" (UID: \"c0b9cf75-f964-4467-ace5-5362020b4be7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"
Jul 02 23:17:13.913678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.913586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0b9cf75-f964-4467-ace5-5362020b4be7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb\" (UID: \"c0b9cf75-f964-4467-ace5-5362020b4be7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"
Jul 02 23:17:13.913678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.913662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cskn4\" (UniqueName: \"kubernetes.io/projected/c0b9cf75-f964-4467-ace5-5362020b4be7-kube-api-access-cskn4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb\" (UID: \"c0b9cf75-f964-4467-ace5-5362020b4be7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"
Jul 02 23:17:13.913916 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.913714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0b9cf75-f964-4467-ace5-5362020b4be7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb\" (UID: \"c0b9cf75-f964-4467-ace5-5362020b4be7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"
Jul 02 23:17:13.913976 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.913948    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0b9cf75-f964-4467-ace5-5362020b4be7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb\" (UID: \"c0b9cf75-f964-4467-ace5-5362020b4be7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"
Jul 02 23:17:13.914058 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.914037    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0b9cf75-f964-4467-ace5-5362020b4be7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb\" (UID: \"c0b9cf75-f964-4467-ace5-5362020b4be7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"
Jul 02 23:17:13.923087 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:13.923061    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cskn4\" (UniqueName: \"kubernetes.io/projected/c0b9cf75-f964-4467-ace5-5362020b4be7-kube-api-access-cskn4\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb\" (UID: \"c0b9cf75-f964-4467-ace5-5362020b4be7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"
Jul 02 23:17:14.075630 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:14.075590    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"
Jul 02 23:17:14.210897 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:14.210871    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"]
Jul 02 23:17:14.273232 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:17:14.273196    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0b9cf75_f964_4467_ace5_5362020b4be7.slice/crio-47edca6b568a70ae01b2eacf36ee0e5ef095204fcb8711c46c4fbe8a5c0fbc97 WatchSource:0}: Error finding container 47edca6b568a70ae01b2eacf36ee0e5ef095204fcb8711c46c4fbe8a5c0fbc97: Status 404 returned error can't find the container with id 47edca6b568a70ae01b2eacf36ee0e5ef095204fcb8711c46c4fbe8a5c0fbc97
Jul 02 23:17:14.787436 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:14.787331    2580 generic.go:356] "Generic (PLEG): container finished" podID="0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb" containerID="dc1f143935df4005e017a0c17679b2ca1cf9f2fcd8c499113dbe4775cdf3cade" exitCode=0
Jul 02 23:17:14.787859 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:14.787425    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g" event={"ID":"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb","Type":"ContainerDied","Data":"dc1f143935df4005e017a0c17679b2ca1cf9f2fcd8c499113dbe4775cdf3cade"}
Jul 02 23:17:14.789589 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:14.789564    2580 generic.go:356] "Generic (PLEG): container finished" podID="2bbc37b4-d84e-44b5-a3f3-86cb542915b0" containerID="e5dd4ddd2a8ac18753c789d64d9392d7921d35391810d16ba6c76a45e807a8f9" exitCode=0
Jul 02 23:17:14.789684 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:14.789646    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn" event={"ID":"2bbc37b4-d84e-44b5-a3f3-86cb542915b0","Type":"ContainerDied","Data":"e5dd4ddd2a8ac18753c789d64d9392d7921d35391810d16ba6c76a45e807a8f9"}
Jul 02 23:17:14.791714 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:14.791691    2580 generic.go:356] "Generic (PLEG): container finished" podID="91527594-f1c6-4f49-bbd0-f862aa1d3477" containerID="6a187a9a41fc27cd12701a778ba8eecc043c84c3256d301204a40c2cd4649053" exitCode=0
Jul 02 23:17:14.791809 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:14.791758    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc" event={"ID":"91527594-f1c6-4f49-bbd0-f862aa1d3477","Type":"ContainerDied","Data":"6a187a9a41fc27cd12701a778ba8eecc043c84c3256d301204a40c2cd4649053"}
Jul 02 23:17:14.793330 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:14.793308    2580 generic.go:356] "Generic (PLEG): container finished" podID="c0b9cf75-f964-4467-ace5-5362020b4be7" containerID="2548b5bfd1fab5ddd59202f2ea0e854d85378f87519709b4e703a55abb687313" exitCode=0
Jul 02 23:17:14.793423 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:14.793355    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb" event={"ID":"c0b9cf75-f964-4467-ace5-5362020b4be7","Type":"ContainerDied","Data":"2548b5bfd1fab5ddd59202f2ea0e854d85378f87519709b4e703a55abb687313"}
Jul 02 23:17:14.793423 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:14.793386    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb" event={"ID":"c0b9cf75-f964-4467-ace5-5362020b4be7","Type":"ContainerStarted","Data":"47edca6b568a70ae01b2eacf36ee0e5ef095204fcb8711c46c4fbe8a5c0fbc97"}
Jul 02 23:17:15.799286 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:15.799247    2580 generic.go:356] "Generic (PLEG): container finished" podID="c0b9cf75-f964-4467-ace5-5362020b4be7" containerID="1f55e792fac0bb6544964aa629dde6eca6471c540d6ea3ba83f476fd81e640da" exitCode=0
Jul 02 23:17:15.799724 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:15.799334    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb" event={"ID":"c0b9cf75-f964-4467-ace5-5362020b4be7","Type":"ContainerDied","Data":"1f55e792fac0bb6544964aa629dde6eca6471c540d6ea3ba83f476fd81e640da"}
Jul 02 23:17:15.801448 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:15.801425    2580 generic.go:356] "Generic (PLEG): container finished" podID="0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb" containerID="d6d69de5ec9314f70f307eb03fc6251fcb940a146ec6c56750a9d83af68c93d1" exitCode=0
Jul 02 23:17:15.801547 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:15.801511    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g" event={"ID":"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb","Type":"ContainerDied","Data":"d6d69de5ec9314f70f307eb03fc6251fcb940a146ec6c56750a9d83af68c93d1"}
Jul 02 23:17:15.960346 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:15.960321    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"
Jul 02 23:17:15.966695 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:15.966669    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"
Jul 02 23:17:16.029882 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.029856    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/91527594-f1c6-4f49-bbd0-f862aa1d3477-util\") pod \"91527594-f1c6-4f49-bbd0-f862aa1d3477\" (UID: \"91527594-f1c6-4f49-bbd0-f862aa1d3477\") "
Jul 02 23:17:16.030054 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.029923    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dlb9\" (UniqueName: \"kubernetes.io/projected/91527594-f1c6-4f49-bbd0-f862aa1d3477-kube-api-access-6dlb9\") pod \"91527594-f1c6-4f49-bbd0-f862aa1d3477\" (UID: \"91527594-f1c6-4f49-bbd0-f862aa1d3477\") "
Jul 02 23:17:16.030054 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.029962    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6lxm\" (UniqueName: \"kubernetes.io/projected/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-kube-api-access-g6lxm\") pod \"2bbc37b4-d84e-44b5-a3f3-86cb542915b0\" (UID: \"2bbc37b4-d84e-44b5-a3f3-86cb542915b0\") "
Jul 02 23:17:16.030054 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.029987    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-bundle\") pod \"2bbc37b4-d84e-44b5-a3f3-86cb542915b0\" (UID: \"2bbc37b4-d84e-44b5-a3f3-86cb542915b0\") "
Jul 02 23:17:16.030054 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.030042    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/91527594-f1c6-4f49-bbd0-f862aa1d3477-bundle\") pod \"91527594-f1c6-4f49-bbd0-f862aa1d3477\" (UID: \"91527594-f1c6-4f49-bbd0-f862aa1d3477\") "
Jul 02 23:17:16.030269 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.030082    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-util\") pod \"2bbc37b4-d84e-44b5-a3f3-86cb542915b0\" (UID: \"2bbc37b4-d84e-44b5-a3f3-86cb542915b0\") "
Jul 02 23:17:16.030714 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.030574    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-bundle" (OuterVolumeSpecName: "bundle") pod "2bbc37b4-d84e-44b5-a3f3-86cb542915b0" (UID: "2bbc37b4-d84e-44b5-a3f3-86cb542915b0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:17:16.030714 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.030667    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91527594-f1c6-4f49-bbd0-f862aa1d3477-bundle" (OuterVolumeSpecName: "bundle") pod "91527594-f1c6-4f49-bbd0-f862aa1d3477" (UID: "91527594-f1c6-4f49-bbd0-f862aa1d3477"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:17:16.032604 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.032584    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91527594-f1c6-4f49-bbd0-f862aa1d3477-kube-api-access-6dlb9" (OuterVolumeSpecName: "kube-api-access-6dlb9") pod "91527594-f1c6-4f49-bbd0-f862aa1d3477" (UID: "91527594-f1c6-4f49-bbd0-f862aa1d3477"). InnerVolumeSpecName "kube-api-access-6dlb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:17:16.032683 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.032615    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-kube-api-access-g6lxm" (OuterVolumeSpecName: "kube-api-access-g6lxm") pod "2bbc37b4-d84e-44b5-a3f3-86cb542915b0" (UID: "2bbc37b4-d84e-44b5-a3f3-86cb542915b0"). InnerVolumeSpecName "kube-api-access-g6lxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:17:16.035648 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.035613    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91527594-f1c6-4f49-bbd0-f862aa1d3477-util" (OuterVolumeSpecName: "util") pod "91527594-f1c6-4f49-bbd0-f862aa1d3477" (UID: "91527594-f1c6-4f49-bbd0-f862aa1d3477"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:17:16.037083 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.037057    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-util" (OuterVolumeSpecName: "util") pod "2bbc37b4-d84e-44b5-a3f3-86cb542915b0" (UID: "2bbc37b4-d84e-44b5-a3f3-86cb542915b0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:17:16.131478 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.131394    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dlb9\" (UniqueName: \"kubernetes.io/projected/91527594-f1c6-4f49-bbd0-f862aa1d3477-kube-api-access-6dlb9\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:16.131478 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.131423    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6lxm\" (UniqueName: \"kubernetes.io/projected/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-kube-api-access-g6lxm\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:16.131478 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.131434    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-bundle\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:16.131478 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.131444    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/91527594-f1c6-4f49-bbd0-f862aa1d3477-bundle\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:16.131478 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.131452    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2bbc37b4-d84e-44b5-a3f3-86cb542915b0-util\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:16.131478 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.131459    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/91527594-f1c6-4f49-bbd0-f862aa1d3477-util\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:16.808164 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.808108    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn" event={"ID":"2bbc37b4-d84e-44b5-a3f3-86cb542915b0","Type":"ContainerDied","Data":"4018b3953cfbd7eb67a821bc204454ff8ed6814e75b256afc65c38d8063879f9"}
Jul 02 23:17:16.808621 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.808186    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4018b3953cfbd7eb67a821bc204454ff8ed6814e75b256afc65c38d8063879f9"
Jul 02 23:17:16.808621 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.808136    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn"
Jul 02 23:17:16.810182 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.810150    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc" event={"ID":"91527594-f1c6-4f49-bbd0-f862aa1d3477","Type":"ContainerDied","Data":"9439bd5c6b659456a221d1707670f3b2ea9075890a4c21899536c4b1bc78fa9d"}
Jul 02 23:17:16.810182 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.810169    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc"
Jul 02 23:17:16.810391 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.810196    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9439bd5c6b659456a221d1707670f3b2ea9075890a4c21899536c4b1bc78fa9d"
Jul 02 23:17:16.812367 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.812344    2580 generic.go:356] "Generic (PLEG): container finished" podID="c0b9cf75-f964-4467-ace5-5362020b4be7" containerID="c6a47069ae1a46e5b195385fc1de54bb6d48628f9a8f259f3e3d3f1c8b70a28f" exitCode=0
Jul 02 23:17:16.812477 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.812414    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb" event={"ID":"c0b9cf75-f964-4467-ace5-5362020b4be7","Type":"ContainerDied","Data":"c6a47069ae1a46e5b195385fc1de54bb6d48628f9a8f259f3e3d3f1c8b70a28f"}
Jul 02 23:17:16.939771 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:16.939746    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"
Jul 02 23:17:17.038276 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:17.038241    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-bundle\") pod \"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb\" (UID: \"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb\") "
Jul 02 23:17:17.038433 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:17.038317    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-util\") pod \"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb\" (UID: \"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb\") "
Jul 02 23:17:17.038433 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:17.038373    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cb9cp\" (UniqueName: \"kubernetes.io/projected/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-kube-api-access-cb9cp\") pod \"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb\" (UID: \"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb\") "
Jul 02 23:17:17.038710 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:17.038678    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-bundle" (OuterVolumeSpecName: "bundle") pod "0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb" (UID: "0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:17:17.040417 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:17.040390    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-kube-api-access-cb9cp" (OuterVolumeSpecName: "kube-api-access-cb9cp") pod "0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb" (UID: "0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb"). InnerVolumeSpecName "kube-api-access-cb9cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:17:17.043546 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:17.043494    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-util" (OuterVolumeSpecName: "util") pod "0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb" (UID: "0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:17:17.138979 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:17.138891    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-bundle\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:17.138979 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:17.138921    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-util\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:17.138979 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:17.138931    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb9cp\" (UniqueName: \"kubernetes.io/projected/0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb-kube-api-access-cb9cp\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:17.818811 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:17.818780    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g"
Jul 02 23:17:17.818811 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:17.818806    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g" event={"ID":"0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb","Type":"ContainerDied","Data":"aad649fe0475bef069277eea0e98fd741a5a704471ccd67e24445e865b511e37"}
Jul 02 23:17:17.819298 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:17.818850    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aad649fe0475bef069277eea0e98fd741a5a704471ccd67e24445e865b511e37"
Jul 02 23:17:17.944109 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:17.944075    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"
Jul 02 23:17:18.045973 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:18.045926    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0b9cf75-f964-4467-ace5-5362020b4be7-util\") pod \"c0b9cf75-f964-4467-ace5-5362020b4be7\" (UID: \"c0b9cf75-f964-4467-ace5-5362020b4be7\") "
Jul 02 23:17:18.046188 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:18.045988    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0b9cf75-f964-4467-ace5-5362020b4be7-bundle\") pod \"c0b9cf75-f964-4467-ace5-5362020b4be7\" (UID: \"c0b9cf75-f964-4467-ace5-5362020b4be7\") "
Jul 02 23:17:18.046188 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:18.046008    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cskn4\" (UniqueName: \"kubernetes.io/projected/c0b9cf75-f964-4467-ace5-5362020b4be7-kube-api-access-cskn4\") pod \"c0b9cf75-f964-4467-ace5-5362020b4be7\" (UID: \"c0b9cf75-f964-4467-ace5-5362020b4be7\") "
Jul 02 23:17:18.046584 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:18.046556    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0b9cf75-f964-4467-ace5-5362020b4be7-bundle" (OuterVolumeSpecName: "bundle") pod "c0b9cf75-f964-4467-ace5-5362020b4be7" (UID: "c0b9cf75-f964-4467-ace5-5362020b4be7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:17:18.048161 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:18.048132    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0b9cf75-f964-4467-ace5-5362020b4be7-kube-api-access-cskn4" (OuterVolumeSpecName: "kube-api-access-cskn4") pod "c0b9cf75-f964-4467-ace5-5362020b4be7" (UID: "c0b9cf75-f964-4467-ace5-5362020b4be7"). InnerVolumeSpecName "kube-api-access-cskn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:17:18.051243 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:18.051208    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0b9cf75-f964-4467-ace5-5362020b4be7-util" (OuterVolumeSpecName: "util") pod "c0b9cf75-f964-4467-ace5-5362020b4be7" (UID: "c0b9cf75-f964-4467-ace5-5362020b4be7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:17:18.147069 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:18.146990    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c0b9cf75-f964-4467-ace5-5362020b4be7-util\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:18.147069 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:18.147016    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c0b9cf75-f964-4467-ace5-5362020b4be7-bundle\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:18.147069 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:18.147024    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cskn4\" (UniqueName: \"kubernetes.io/projected/c0b9cf75-f964-4467-ace5-5362020b4be7-kube-api-access-cskn4\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:18.825358 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:18.825321    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb"
Jul 02 23:17:18.825358 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:18.825328    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb" event={"ID":"c0b9cf75-f964-4467-ace5-5362020b4be7","Type":"ContainerDied","Data":"47edca6b568a70ae01b2eacf36ee0e5ef095204fcb8711c46c4fbe8a5c0fbc97"}
Jul 02 23:17:18.825912 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:18.825396    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47edca6b568a70ae01b2eacf36ee0e5ef095204fcb8711c46c4fbe8a5c0fbc97"
Jul 02 23:17:24.919312 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919279    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"]
Jul 02 23:17:24.919678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919593    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb" containerName="extract"
Jul 02 23:17:24.919678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919608    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb" containerName="extract"
Jul 02 23:17:24.919678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919627    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0b9cf75-f964-4467-ace5-5362020b4be7" containerName="extract"
Jul 02 23:17:24.919678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919633    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0b9cf75-f964-4467-ace5-5362020b4be7" containerName="extract"
Jul 02 23:17:24.919678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919641    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0b9cf75-f964-4467-ace5-5362020b4be7" containerName="util"
Jul 02 23:17:24.919678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919647    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0b9cf75-f964-4467-ace5-5362020b4be7" containerName="util"
Jul 02 23:17:24.919678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919653    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb" containerName="util"
Jul 02 23:17:24.919678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919658    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb" containerName="util"
Jul 02 23:17:24.919678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919665    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bbc37b4-d84e-44b5-a3f3-86cb542915b0" containerName="extract"
Jul 02 23:17:24.919678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919673    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bbc37b4-d84e-44b5-a3f3-86cb542915b0" containerName="extract"
Jul 02 23:17:24.919678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919680    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0b9cf75-f964-4467-ace5-5362020b4be7" containerName="pull"
Jul 02 23:17:24.919678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919685    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0b9cf75-f964-4467-ace5-5362020b4be7" containerName="pull"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919691    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91527594-f1c6-4f49-bbd0-f862aa1d3477" containerName="util"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919696    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="91527594-f1c6-4f49-bbd0-f862aa1d3477" containerName="util"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919701    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bbc37b4-d84e-44b5-a3f3-86cb542915b0" containerName="pull"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919705    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bbc37b4-d84e-44b5-a3f3-86cb542915b0" containerName="pull"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919711    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91527594-f1c6-4f49-bbd0-f862aa1d3477" containerName="extract"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919716    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="91527594-f1c6-4f49-bbd0-f862aa1d3477" containerName="extract"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919723    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91527594-f1c6-4f49-bbd0-f862aa1d3477" containerName="pull"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919727    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="91527594-f1c6-4f49-bbd0-f862aa1d3477" containerName="pull"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919735    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bbc37b4-d84e-44b5-a3f3-86cb542915b0" containerName="util"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919741    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bbc37b4-d84e-44b5-a3f3-86cb542915b0" containerName="util"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919750    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb" containerName="pull"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919758    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb" containerName="pull"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919798    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="91527594-f1c6-4f49-bbd0-f862aa1d3477" containerName="extract"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919806    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bbc37b4-d84e-44b5-a3f3-86cb542915b0" containerName="extract"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919812    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb" containerName="extract"
Jul 02 23:17:24.920027 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.919820    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0b9cf75-f964-4467-ace5-5362020b4be7" containerName="extract"
Jul 02 23:17:24.922742 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.922719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:24.926193 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.926170    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rh4ws\""
Jul 02 23:17:24.949588 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.949550    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"]
Jul 02 23:17:24.999631 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.999593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pg2m\" (UniqueName: \"kubernetes.io/projected/10d28a77-5aef-415b-84c9-0582703cefb8-kube-api-access-8pg2m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" (UID: \"10d28a77-5aef-415b-84c9-0582703cefb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:24.999814 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:24.999663    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10d28a77-5aef-415b-84c9-0582703cefb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" (UID: \"10d28a77-5aef-415b-84c9-0582703cefb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:25.100284 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:25.100251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10d28a77-5aef-415b-84c9-0582703cefb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" (UID: \"10d28a77-5aef-415b-84c9-0582703cefb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:25.100454 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:25.100314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pg2m\" (UniqueName: \"kubernetes.io/projected/10d28a77-5aef-415b-84c9-0582703cefb8-kube-api-access-8pg2m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" (UID: \"10d28a77-5aef-415b-84c9-0582703cefb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:25.100616 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:25.100597    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10d28a77-5aef-415b-84c9-0582703cefb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" (UID: \"10d28a77-5aef-415b-84c9-0582703cefb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:25.117555 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:25.117525    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pg2m\" (UniqueName: \"kubernetes.io/projected/10d28a77-5aef-415b-84c9-0582703cefb8-kube-api-access-8pg2m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" (UID: \"10d28a77-5aef-415b-84c9-0582703cefb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:25.232425 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:25.232393    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:25.379549 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:25.379516    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"]
Jul 02 23:17:25.382793 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:17:25.382767    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d28a77_5aef_415b_84c9_0582703cefb8.slice/crio-d504cf60366fe3ab73175776ae5f43456e2172f35a013844d4294b61c740f1ed WatchSource:0}: Error finding container d504cf60366fe3ab73175776ae5f43456e2172f35a013844d4294b61c740f1ed: Status 404 returned error can't find the container with id d504cf60366fe3ab73175776ae5f43456e2172f35a013844d4294b61c740f1ed
Jul 02 23:17:25.856992 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:25.856947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f" event={"ID":"10d28a77-5aef-415b-84c9-0582703cefb8","Type":"ContainerStarted","Data":"d504cf60366fe3ab73175776ae5f43456e2172f35a013844d4294b61c740f1ed"}
Jul 02 23:17:27.832797 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:27.832760    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vwf7m"]
Jul 02 23:17:27.836312 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:27.836290    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vwf7m"
Jul 02 23:17:27.852365 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:27.852342    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-rtf82\""
Jul 02 23:17:27.894364 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:27.894335    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vwf7m"]
Jul 02 23:17:27.922873 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:27.922845    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcw2f\" (UniqueName: \"kubernetes.io/projected/8aa3ce58-a92d-4963-bb7f-d81c64a010d8-kube-api-access-jcw2f\") pod \"authorino-operator-75cc8d5cb7-vwf7m\" (UID: \"8aa3ce58-a92d-4963-bb7f-d81c64a010d8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vwf7m"
Jul 02 23:17:28.023463 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:28.023418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcw2f\" (UniqueName: \"kubernetes.io/projected/8aa3ce58-a92d-4963-bb7f-d81c64a010d8-kube-api-access-jcw2f\") pod \"authorino-operator-75cc8d5cb7-vwf7m\" (UID: \"8aa3ce58-a92d-4963-bb7f-d81c64a010d8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vwf7m"
Jul 02 23:17:28.037177 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:28.037148    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcw2f\" (UniqueName: \"kubernetes.io/projected/8aa3ce58-a92d-4963-bb7f-d81c64a010d8-kube-api-access-jcw2f\") pod \"authorino-operator-75cc8d5cb7-vwf7m\" (UID: \"8aa3ce58-a92d-4963-bb7f-d81c64a010d8\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-vwf7m"
Jul 02 23:17:28.146504 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:28.146412    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vwf7m"
Jul 02 23:17:29.225363 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:29.225340    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-vwf7m"]
Jul 02 23:17:29.877645 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:29.877597    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vwf7m" event={"ID":"8aa3ce58-a92d-4963-bb7f-d81c64a010d8","Type":"ContainerStarted","Data":"32e0215ba656aadae87354bb0c3b1acf175f5e9f68696652e3111b97f7add97b"}
Jul 02 23:17:29.879760 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:29.879720    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f" event={"ID":"10d28a77-5aef-415b-84c9-0582703cefb8","Type":"ContainerStarted","Data":"db6bae242413599d0ecbf4dc2d2e692cfd32c79101929625af745107cd25dd16"}
Jul 02 23:17:29.879910 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:29.879861    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:29.994008 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:29.993947    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f" podStartSLOduration=2.2384487220000002 podStartE2EDuration="5.993923901s" totalImagesPullingTime="3.755475179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:17:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:17:29.940315708 +0000 UTC m=+395.194105710" watchObservedRunningTime="2026-07-02 23:17:29.993923901 +0000 UTC m=+395.247713898"
Jul 02 23:17:29.996178 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:29.996146    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cj4rk"]
Jul 02 23:17:29.999828 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:29.999801    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cj4rk"
Jul 02 23:17:30.005590 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:30.005564    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5jqbq\""
Jul 02 23:17:30.005780 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:30.005758    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 02 23:17:30.025048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:30.024350    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cj4rk"]
Jul 02 23:17:30.040896 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:30.040863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc8kl\" (UniqueName: \"kubernetes.io/projected/292cfb62-98cd-426c-aab9-9d28ec65db6d-kube-api-access-pc8kl\") pod \"dns-operator-controller-manager-864dd78c4c-cj4rk\" (UID: \"292cfb62-98cd-426c-aab9-9d28ec65db6d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cj4rk"
Jul 02 23:17:30.142077 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:30.141989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc8kl\" (UniqueName: \"kubernetes.io/projected/292cfb62-98cd-426c-aab9-9d28ec65db6d-kube-api-access-pc8kl\") pod \"dns-operator-controller-manager-864dd78c4c-cj4rk\" (UID: \"292cfb62-98cd-426c-aab9-9d28ec65db6d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cj4rk"
Jul 02 23:17:30.163955 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:30.163925    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc8kl\" (UniqueName: \"kubernetes.io/projected/292cfb62-98cd-426c-aab9-9d28ec65db6d-kube-api-access-pc8kl\") pod \"dns-operator-controller-manager-864dd78c4c-cj4rk\" (UID: \"292cfb62-98cd-426c-aab9-9d28ec65db6d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cj4rk"
Jul 02 23:17:30.314988 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:30.314953    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cj4rk"
Jul 02 23:17:31.296719 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:31.296696    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cj4rk"]
Jul 02 23:17:31.298376 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:17:31.298352    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod292cfb62_98cd_426c_aab9_9d28ec65db6d.slice/crio-3af3bb83c0bec3ea807ad6eced969b7dfa5e1777a0019d3da6457eabae65c815 WatchSource:0}: Error finding container 3af3bb83c0bec3ea807ad6eced969b7dfa5e1777a0019d3da6457eabae65c815: Status 404 returned error can't find the container with id 3af3bb83c0bec3ea807ad6eced969b7dfa5e1777a0019d3da6457eabae65c815
Jul 02 23:17:31.891101 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:31.891056    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vwf7m" event={"ID":"8aa3ce58-a92d-4963-bb7f-d81c64a010d8","Type":"ContainerStarted","Data":"c3fc611430e6cb1fda8dff34f2759b1f09334cb44ca75d31776a1b2f801c1bdc"}
Jul 02 23:17:31.891555 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:31.891299    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vwf7m"
Jul 02 23:17:31.892443 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:31.892405    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cj4rk" event={"ID":"292cfb62-98cd-426c-aab9-9d28ec65db6d","Type":"ContainerStarted","Data":"3af3bb83c0bec3ea807ad6eced969b7dfa5e1777a0019d3da6457eabae65c815"}
Jul 02 23:17:31.923183 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:31.923111    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vwf7m" podStartSLOduration=2.95213264 podStartE2EDuration="4.923094043s" totalImagesPullingTime="1.970961403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:17:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:17:31.922556047 +0000 UTC m=+397.176346046" watchObservedRunningTime="2026-07-02 23:17:31.923094043 +0000 UTC m=+397.176884042"
Jul 02 23:17:32.364194 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.364154    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"]
Jul 02 23:17:32.367344 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.367319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:32.378359 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.378330    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-57wj6\""
Jul 02 23:17:32.391632 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.391604    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"]
Jul 02 23:17:32.453080 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.453043    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd"]
Jul 02 23:17:32.456152 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.456133    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd"
Jul 02 23:17:32.459736 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.459714    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 02 23:17:32.459871 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.459742    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-22wxt\""
Jul 02 23:17:32.460000 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.459983    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 02 23:17:32.460128 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.460020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjpps\" (UniqueName: \"kubernetes.io/projected/82824644-1af7-4b0c-b220-8702d2df4e81-kube-api-access-fjpps\") pod \"limitador-operator-controller-manager-859d7b78bd-mxspm\" (UID: \"82824644-1af7-4b0c-b220-8702d2df4e81\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:32.483853 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.483819    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd"]
Jul 02 23:17:32.561018 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.560985    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9850397a-ee60-40de-9e65-6b3ced3e0194-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wqjwd\" (UID: \"9850397a-ee60-40de-9e65-6b3ced3e0194\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd"
Jul 02 23:17:32.561232 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.561026    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9850397a-ee60-40de-9e65-6b3ced3e0194-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wqjwd\" (UID: \"9850397a-ee60-40de-9e65-6b3ced3e0194\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd"
Jul 02 23:17:32.561232 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.561062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtl55\" (UniqueName: \"kubernetes.io/projected/9850397a-ee60-40de-9e65-6b3ced3e0194-kube-api-access-xtl55\") pod \"kuadrant-console-plugin-66b5774985-wqjwd\" (UID: \"9850397a-ee60-40de-9e65-6b3ced3e0194\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd"
Jul 02 23:17:32.561361 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.561256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjpps\" (UniqueName: \"kubernetes.io/projected/82824644-1af7-4b0c-b220-8702d2df4e81-kube-api-access-fjpps\") pod \"limitador-operator-controller-manager-859d7b78bd-mxspm\" (UID: \"82824644-1af7-4b0c-b220-8702d2df4e81\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:32.576128 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.576085    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjpps\" (UniqueName: \"kubernetes.io/projected/82824644-1af7-4b0c-b220-8702d2df4e81-kube-api-access-fjpps\") pod \"limitador-operator-controller-manager-859d7b78bd-mxspm\" (UID: \"82824644-1af7-4b0c-b220-8702d2df4e81\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:32.662156 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.662063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9850397a-ee60-40de-9e65-6b3ced3e0194-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wqjwd\" (UID: \"9850397a-ee60-40de-9e65-6b3ced3e0194\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd"
Jul 02 23:17:32.662156 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.662103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9850397a-ee60-40de-9e65-6b3ced3e0194-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wqjwd\" (UID: \"9850397a-ee60-40de-9e65-6b3ced3e0194\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd"
Jul 02 23:17:32.662156 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.662147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtl55\" (UniqueName: \"kubernetes.io/projected/9850397a-ee60-40de-9e65-6b3ced3e0194-kube-api-access-xtl55\") pod \"kuadrant-console-plugin-66b5774985-wqjwd\" (UID: \"9850397a-ee60-40de-9e65-6b3ced3e0194\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd"
Jul 02 23:17:32.662900 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.662872    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9850397a-ee60-40de-9e65-6b3ced3e0194-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-wqjwd\" (UID: \"9850397a-ee60-40de-9e65-6b3ced3e0194\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd"
Jul 02 23:17:32.664893 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.664866    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9850397a-ee60-40de-9e65-6b3ced3e0194-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-wqjwd\" (UID: \"9850397a-ee60-40de-9e65-6b3ced3e0194\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd"
Jul 02 23:17:32.674353 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.674327    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtl55\" (UniqueName: \"kubernetes.io/projected/9850397a-ee60-40de-9e65-6b3ced3e0194-kube-api-access-xtl55\") pod \"kuadrant-console-plugin-66b5774985-wqjwd\" (UID: \"9850397a-ee60-40de-9e65-6b3ced3e0194\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd"
Jul 02 23:17:32.681938 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.681909    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:32.765726 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:32.765691    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd"
Jul 02 23:17:33.091965 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:33.091936    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"]
Jul 02 23:17:33.094288 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:17:33.094262    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82824644_1af7_4b0c_b220_8702d2df4e81.slice/crio-c78d2c8eea716a76c7df68406ff960b208d435d4a40c0d58ac1416e2cd74e908 WatchSource:0}: Error finding container c78d2c8eea716a76c7df68406ff960b208d435d4a40c0d58ac1416e2cd74e908: Status 404 returned error can't find the container with id c78d2c8eea716a76c7df68406ff960b208d435d4a40c0d58ac1416e2cd74e908
Jul 02 23:17:33.110830 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:33.110806    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd"]
Jul 02 23:17:33.112474 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:17:33.112453    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9850397a_ee60_40de_9e65_6b3ced3e0194.slice/crio-49122e06f03631da51b57510d0e00f5550029724dfa566bb93a4a8c6a03a1cb1 WatchSource:0}: Error finding container 49122e06f03631da51b57510d0e00f5550029724dfa566bb93a4a8c6a03a1cb1: Status 404 returned error can't find the container with id 49122e06f03631da51b57510d0e00f5550029724dfa566bb93a4a8c6a03a1cb1
Jul 02 23:17:33.906243 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:33.906199    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cj4rk" event={"ID":"292cfb62-98cd-426c-aab9-9d28ec65db6d","Type":"ContainerStarted","Data":"9ab54d7a46d60ad2f2f467fbfab6f428c2c9323cf09ff4e8daa5c08e59c92c32"}
Jul 02 23:17:33.906612 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:33.906574    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cj4rk"
Jul 02 23:17:33.907994 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:33.907961    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd" event={"ID":"9850397a-ee60-40de-9e65-6b3ced3e0194","Type":"ContainerStarted","Data":"49122e06f03631da51b57510d0e00f5550029724dfa566bb93a4a8c6a03a1cb1"}
Jul 02 23:17:33.909303 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:33.909277    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm" event={"ID":"82824644-1af7-4b0c-b220-8702d2df4e81","Type":"ContainerStarted","Data":"c78d2c8eea716a76c7df68406ff960b208d435d4a40c0d58ac1416e2cd74e908"}
Jul 02 23:17:33.967942 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:33.967880    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cj4rk" podStartSLOduration=3.267678188 podStartE2EDuration="4.96785782s" totalImagesPullingTime="1.700179632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:17:33.965483222 +0000 UTC m=+399.219273221" watchObservedRunningTime="2026-07-02 23:17:33.96785782 +0000 UTC m=+399.221647820"
Jul 02 23:17:34.917020 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:34.916968    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm" event={"ID":"82824644-1af7-4b0c-b220-8702d2df4e81","Type":"ContainerStarted","Data":"d52cd3f0c020aec92d9efa1466f117ac2e419bb314b776f48821a1fb6e604c72"}
Jul 02 23:17:34.917493 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:34.917156    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:34.952510 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:34.952454    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm" podStartSLOduration=1.629506434 podStartE2EDuration="2.95243827s" totalImagesPullingTime="1.322931836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:17:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:17:34.949546025 +0000 UTC m=+400.203336024" watchObservedRunningTime="2026-07-02 23:17:34.95243827 +0000 UTC m=+400.206228269"
Jul 02 23:17:40.887690 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:40.887654    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:42.763349 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.763315    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"]
Jul 02 23:17:42.801050 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.801017    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"]
Jul 02 23:17:42.801244 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.801180    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"
Jul 02 23:17:42.870491 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.870453    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"]
Jul 02 23:17:42.870697 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.870670    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" containerName="manager" containerID="cri-o://db6bae242413599d0ecbf4dc2d2e692cfd32c79101929625af745107cd25dd16" gracePeriod=2
Jul 02 23:17:42.884083 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.884051    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"]
Jul 02 23:17:42.891794 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:42.891754    2580 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"10d28a77-5aef-415b-84c9-0582703cefb8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-4828f\": pods \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:42.901167 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.901139    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-vwf7m"
Jul 02 23:17:42.903884 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.903535    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"]
Jul 02 23:17:42.904020 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.903991    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" containerName="manager"
Jul 02 23:17:42.904020 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.904009    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" containerName="manager"
Jul 02 23:17:42.904146 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.904106    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" containerName="manager"
Jul 02 23:17:42.909475 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.909448    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"]
Jul 02 23:17:42.909620 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.909548    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"
Jul 02 23:17:42.909858 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:42.909834    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-v566s], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks" podUID="df1ad485-24f8-486f-9127-d13178a9e419"
Jul 02 23:17:42.918589 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.918526    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"]
Jul 02 23:17:42.941379 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.941345    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"]
Jul 02 23:17:42.944283 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.944261    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"]
Jul 02 23:17:42.944708 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.944654    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" containerName="manager" containerID="cri-o://d52cd3f0c020aec92d9efa1466f117ac2e419bb314b776f48821a1fb6e604c72" gracePeriod=2
Jul 02 23:17:42.948537 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.948461    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:42.950036 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:42.950002    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:42.950707 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.950681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df1ad485-24f8-486f-9127-d13178a9e419-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d7nks\" (UID: \"df1ad485-24f8-486f-9127-d13178a9e419\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"
Jul 02 23:17:42.950886 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.950869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v566s\" (UniqueName: \"kubernetes.io/projected/df1ad485-24f8-486f-9127-d13178a9e419-kube-api-access-v566s\") pod \"kuadrant-operator-controller-manager-86878c769f-d7nks\" (UID: \"df1ad485-24f8-486f-9127-d13178a9e419\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"
Jul 02 23:17:42.953322 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:42.953291    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mxspm\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:42.953744 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.953722    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"]
Jul 02 23:17:42.964299 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.963635    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"
Jul 02 23:17:42.988184 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.988130    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wsg8r"]
Jul 02 23:17:42.988812 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.988682    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" containerName="manager"
Jul 02 23:17:42.988812 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.988704    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" containerName="manager"
Jul 02 23:17:42.988812 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.988774    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" containerName="manager"
Jul 02 23:17:42.989951 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:42.989740    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:42.992724 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:42.992690    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-d7nks\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="df1ad485-24f8-486f-9127-d13178a9e419" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"
Jul 02 23:17:42.994008 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:42.993979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wsg8r"
Jul 02 23:17:42.995004 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:42.994979    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mxspm\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:42.997416 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:42.997393    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:43.006038 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.006015    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wsg8r"]
Jul 02 23:17:43.019974 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.019913    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"
Jul 02 23:17:43.020513 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:43.020376    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mxspm\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:43.022982 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:43.022818    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:43.025552 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:43.025522    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-d7nks\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="df1ad485-24f8-486f-9127-d13178a9e419" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"
Jul 02 23:17:43.028811 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:43.028787    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mxspm\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:43.031230 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:43.031203    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:43.033401 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:43.033374    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-d7nks\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="df1ad485-24f8-486f-9127-d13178a9e419" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"
Jul 02 23:17:43.051812 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.051772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v566s\" (UniqueName: \"kubernetes.io/projected/df1ad485-24f8-486f-9127-d13178a9e419-kube-api-access-v566s\") pod \"kuadrant-operator-controller-manager-86878c769f-d7nks\" (UID: \"df1ad485-24f8-486f-9127-d13178a9e419\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"
Jul 02 23:17:43.051812 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.051816    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04a7240a-1b6b-49f9-8911-908bfac52ae1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zw9cc\" (UID: \"04a7240a-1b6b-49f9-8911-908bfac52ae1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"
Jul 02 23:17:43.052049 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.051853    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmr8k\" (UniqueName: \"kubernetes.io/projected/04a7240a-1b6b-49f9-8911-908bfac52ae1-kube-api-access-zmr8k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zw9cc\" (UID: \"04a7240a-1b6b-49f9-8911-908bfac52ae1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"
Jul 02 23:17:43.052049 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.051962    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df1ad485-24f8-486f-9127-d13178a9e419-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d7nks\" (UID: \"df1ad485-24f8-486f-9127-d13178a9e419\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"
Jul 02 23:17:43.052370 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.052350    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df1ad485-24f8-486f-9127-d13178a9e419-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d7nks\" (UID: \"df1ad485-24f8-486f-9127-d13178a9e419\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"
Jul 02 23:17:43.058167 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:43.057831    2580 projected.go:196] Error preparing data for projected volume kube-api-access-v566s for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-d7nks" not found
Jul 02 23:17:43.058167 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:43.057928    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/df1ad485-24f8-486f-9127-d13178a9e419-kube-api-access-v566s podName:df1ad485-24f8-486f-9127-d13178a9e419 nodeName:}" failed. No retries permitted until 2026-07-02 23:17:43.557905561 +0000 UTC m=+408.811695552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-v566s" (UniqueName: "kubernetes.io/projected/df1ad485-24f8-486f-9127-d13178a9e419-kube-api-access-v566s") pod "kuadrant-operator-controller-manager-86878c769f-d7nks" (UID: "df1ad485-24f8-486f-9127-d13178a9e419") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-d7nks" not found
Jul 02 23:17:43.152969 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.152926    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df1ad485-24f8-486f-9127-d13178a9e419-extensions-socket-volume\") pod \"df1ad485-24f8-486f-9127-d13178a9e419\" (UID: \"df1ad485-24f8-486f-9127-d13178a9e419\") "
Jul 02 23:17:43.153215 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.153178    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmr8k\" (UniqueName: \"kubernetes.io/projected/04a7240a-1b6b-49f9-8911-908bfac52ae1-kube-api-access-zmr8k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zw9cc\" (UID: \"04a7240a-1b6b-49f9-8911-908bfac52ae1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"
Jul 02 23:17:43.153343 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.153206    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df1ad485-24f8-486f-9127-d13178a9e419-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "df1ad485-24f8-486f-9127-d13178a9e419" (UID: "df1ad485-24f8-486f-9127-d13178a9e419"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:17:43.153343 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.153293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6kbf\" (UniqueName: \"kubernetes.io/projected/cc0ea9df-b88b-4bf6-9f29-d02f6a53f198-kube-api-access-g6kbf\") pod \"limitador-operator-controller-manager-859d7b78bd-wsg8r\" (UID: \"cc0ea9df-b88b-4bf6-9f29-d02f6a53f198\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wsg8r"
Jul 02 23:17:43.153477 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.153389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04a7240a-1b6b-49f9-8911-908bfac52ae1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zw9cc\" (UID: \"04a7240a-1b6b-49f9-8911-908bfac52ae1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"
Jul 02 23:17:43.153477 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.153452    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df1ad485-24f8-486f-9127-d13178a9e419-extensions-socket-volume\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:43.153477 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.153469    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v566s\" (UniqueName: \"kubernetes.io/projected/df1ad485-24f8-486f-9127-d13178a9e419-kube-api-access-v566s\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:43.153826 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.153809    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04a7240a-1b6b-49f9-8911-908bfac52ae1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zw9cc\" (UID: \"04a7240a-1b6b-49f9-8911-908bfac52ae1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"
Jul 02 23:17:43.165417 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.165384    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmr8k\" (UniqueName: \"kubernetes.io/projected/04a7240a-1b6b-49f9-8911-908bfac52ae1-kube-api-access-zmr8k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zw9cc\" (UID: \"04a7240a-1b6b-49f9-8911-908bfac52ae1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"
Jul 02 23:17:43.252190 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.252156    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"
Jul 02 23:17:43.254486 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.254450    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6kbf\" (UniqueName: \"kubernetes.io/projected/cc0ea9df-b88b-4bf6-9f29-d02f6a53f198-kube-api-access-g6kbf\") pod \"limitador-operator-controller-manager-859d7b78bd-wsg8r\" (UID: \"cc0ea9df-b88b-4bf6-9f29-d02f6a53f198\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wsg8r"
Jul 02 23:17:43.272571 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.272499    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6kbf\" (UniqueName: \"kubernetes.io/projected/cc0ea9df-b88b-4bf6-9f29-d02f6a53f198-kube-api-access-g6kbf\") pod \"limitador-operator-controller-manager-859d7b78bd-wsg8r\" (UID: \"cc0ea9df-b88b-4bf6-9f29-d02f6a53f198\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wsg8r"
Jul 02 23:17:43.283890 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.283861    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df1ad485-24f8-486f-9127-d13178a9e419" path="/var/lib/kubelet/pods/df1ad485-24f8-486f-9127-d13178a9e419/volumes"
Jul 02 23:17:43.327536 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.327496    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wsg8r"
Jul 02 23:17:43.969874 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:43.969461    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"
Jul 02 23:17:43.974845 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:43.974808    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-d7nks\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="df1ad485-24f8-486f-9127-d13178a9e419" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"
Jul 02 23:17:44.919793 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:44.919761    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cj4rk"
Jul 02 23:17:44.922434 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:44.922403    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-d7nks\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="df1ad485-24f8-486f-9127-d13178a9e419" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"
Jul 02 23:17:45.283459 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:45.283418    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-d7nks\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="df1ad485-24f8-486f-9127-d13178a9e419" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d7nks"
Jul 02 23:17:53.928416 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:53.928383    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wsg8r"]
Jul 02 23:17:53.929486 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:53.929463    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:53.929868 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:17:53.929846    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc0ea9df_b88b_4bf6_9f29_d02f6a53f198.slice/crio-c16dc747d6b6d4e740c814f6aa15a86b5c1e99d36d93b032754b6b3462d47af6 WatchSource:0}: Error finding container c16dc747d6b6d4e740c814f6aa15a86b5c1e99d36d93b032754b6b3462d47af6: Status 404 returned error can't find the container with id c16dc747d6b6d4e740c814f6aa15a86b5c1e99d36d93b032754b6b3462d47af6
Jul 02 23:17:53.931817 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:53.931797    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:53.933037 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:53.933010    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mxspm\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:53.940483 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:53.940458    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:53.943715 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:53.943683    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mxspm\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:53.949458 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:53.949434    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"]
Jul 02 23:17:53.950991 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:17:53.950969    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a7240a_1b6b_49f9_8911_908bfac52ae1.slice/crio-b5101315b7bc66a9a78a3b6fcc65936cbf748072849848b92f609032193745a2 WatchSource:0}: Error finding container b5101315b7bc66a9a78a3b6fcc65936cbf748072849848b92f609032193745a2: Status 404 returned error can't find the container with id b5101315b7bc66a9a78a3b6fcc65936cbf748072849848b92f609032193745a2
Jul 02 23:17:54.021415 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.021383    2580 generic.go:356] "Generic (PLEG): container finished" podID="82824644-1af7-4b0c-b220-8702d2df4e81" containerID="d52cd3f0c020aec92d9efa1466f117ac2e419bb314b776f48821a1fb6e604c72" exitCode=0
Jul 02 23:17:54.021547 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.021424    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:54.021547 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.021449    2580 scope.go:117] "RemoveContainer" containerID="d52cd3f0c020aec92d9efa1466f117ac2e419bb314b776f48821a1fb6e604c72"
Jul 02 23:17:54.025362 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.023070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc" event={"ID":"04a7240a-1b6b-49f9-8911-908bfac52ae1","Type":"ContainerStarted","Data":"44fff592b5e4304fe69d177e557a2c9ac5258086de7f5be3f19419acfee1a652"}
Jul 02 23:17:54.025362 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.023499    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc" event={"ID":"04a7240a-1b6b-49f9-8911-908bfac52ae1","Type":"ContainerStarted","Data":"b5101315b7bc66a9a78a3b6fcc65936cbf748072849848b92f609032193745a2"}
Jul 02 23:17:54.025362 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.023635    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"
Jul 02 23:17:54.025362 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:54.024620    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:54.026946 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:54.026882    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mxspm\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:54.029421 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.027165    2580 generic.go:356] "Generic (PLEG): container finished" podID="10d28a77-5aef-415b-84c9-0582703cefb8" containerID="db6bae242413599d0ecbf4dc2d2e692cfd32c79101929625af745107cd25dd16" exitCode=0
Jul 02 23:17:54.029421 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.027311    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:54.029421 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:54.029141    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:54.031442 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.031412    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd" event={"ID":"9850397a-ee60-40de-9e65-6b3ced3e0194","Type":"ContainerStarted","Data":"d2b0876bd1df7eb1ec33303459e51ab7cb1672fb3adc98f90c5a08c7d7d3ac6b"}
Jul 02 23:17:54.031529 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:54.031411    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mxspm\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:54.033172 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.033146    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wsg8r" event={"ID":"cc0ea9df-b88b-4bf6-9f29-d02f6a53f198","Type":"ContainerStarted","Data":"4cb3240a8b803b3870ab2ed787d8c83c991f360dce2a08c393969fa1f9e4227e"}
Jul 02 23:17:54.033265 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.033226    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wsg8r" event={"ID":"cc0ea9df-b88b-4bf6-9f29-d02f6a53f198","Type":"ContainerStarted","Data":"c16dc747d6b6d4e740c814f6aa15a86b5c1e99d36d93b032754b6b3462d47af6"}
Jul 02 23:17:54.033586 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.033571    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wsg8r"
Jul 02 23:17:54.035047 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.035028    2580 scope.go:117] "RemoveContainer" containerID="d52cd3f0c020aec92d9efa1466f117ac2e419bb314b776f48821a1fb6e604c72"
Jul 02 23:17:54.035349 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:54.035320    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d52cd3f0c020aec92d9efa1466f117ac2e419bb314b776f48821a1fb6e604c72\": container with ID starting with d52cd3f0c020aec92d9efa1466f117ac2e419bb314b776f48821a1fb6e604c72 not found: ID does not exist" containerID="d52cd3f0c020aec92d9efa1466f117ac2e419bb314b776f48821a1fb6e604c72"
Jul 02 23:17:54.035409 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.035361    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d52cd3f0c020aec92d9efa1466f117ac2e419bb314b776f48821a1fb6e604c72"} err="failed to get container status \"d52cd3f0c020aec92d9efa1466f117ac2e419bb314b776f48821a1fb6e604c72\": rpc error: code = NotFound desc = could not find container \"d52cd3f0c020aec92d9efa1466f117ac2e419bb314b776f48821a1fb6e604c72\": container with ID starting with d52cd3f0c020aec92d9efa1466f117ac2e419bb314b776f48821a1fb6e604c72 not found: ID does not exist"
Jul 02 23:17:54.035409 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.035381    2580 scope.go:117] "RemoveContainer" containerID="db6bae242413599d0ecbf4dc2d2e692cfd32c79101929625af745107cd25dd16"
Jul 02 23:17:54.043421 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.043399    2580 scope.go:117] "RemoveContainer" containerID="db6bae242413599d0ecbf4dc2d2e692cfd32c79101929625af745107cd25dd16"
Jul 02 23:17:54.043698 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:54.043677    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db6bae242413599d0ecbf4dc2d2e692cfd32c79101929625af745107cd25dd16\": container with ID starting with db6bae242413599d0ecbf4dc2d2e692cfd32c79101929625af745107cd25dd16 not found: ID does not exist" containerID="db6bae242413599d0ecbf4dc2d2e692cfd32c79101929625af745107cd25dd16"
Jul 02 23:17:54.043762 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.043706    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db6bae242413599d0ecbf4dc2d2e692cfd32c79101929625af745107cd25dd16"} err="failed to get container status \"db6bae242413599d0ecbf4dc2d2e692cfd32c79101929625af745107cd25dd16\": rpc error: code = NotFound desc = could not find container \"db6bae242413599d0ecbf4dc2d2e692cfd32c79101929625af745107cd25dd16\": container with ID starting with db6bae242413599d0ecbf4dc2d2e692cfd32c79101929625af745107cd25dd16 not found: ID does not exist"
Jul 02 23:17:54.052070 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.052043    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pg2m\" (UniqueName: \"kubernetes.io/projected/10d28a77-5aef-415b-84c9-0582703cefb8-kube-api-access-8pg2m\") pod \"10d28a77-5aef-415b-84c9-0582703cefb8\" (UID: \"10d28a77-5aef-415b-84c9-0582703cefb8\") "
Jul 02 23:17:54.052209 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.052091    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjpps\" (UniqueName: \"kubernetes.io/projected/82824644-1af7-4b0c-b220-8702d2df4e81-kube-api-access-fjpps\") pod \"82824644-1af7-4b0c-b220-8702d2df4e81\" (UID: \"82824644-1af7-4b0c-b220-8702d2df4e81\") "
Jul 02 23:17:54.052209 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.052140    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10d28a77-5aef-415b-84c9-0582703cefb8-extensions-socket-volume\") pod \"10d28a77-5aef-415b-84c9-0582703cefb8\" (UID: \"10d28a77-5aef-415b-84c9-0582703cefb8\") "
Jul 02 23:17:54.052763 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.052731    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10d28a77-5aef-415b-84c9-0582703cefb8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "10d28a77-5aef-415b-84c9-0582703cefb8" (UID: "10d28a77-5aef-415b-84c9-0582703cefb8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:17:54.054311 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.054259    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc" podStartSLOduration=12.05424544 podStartE2EDuration="12.05424544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:17:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:17:54.052762286 +0000 UTC m=+419.306552287" watchObservedRunningTime="2026-07-02 23:17:54.05424544 +0000 UTC m=+419.308035438"
Jul 02 23:17:54.054583 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.054556    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10d28a77-5aef-415b-84c9-0582703cefb8-kube-api-access-8pg2m" (OuterVolumeSpecName: "kube-api-access-8pg2m") pod "10d28a77-5aef-415b-84c9-0582703cefb8" (UID: "10d28a77-5aef-415b-84c9-0582703cefb8"). InnerVolumeSpecName "kube-api-access-8pg2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:17:54.054676 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.054658    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82824644-1af7-4b0c-b220-8702d2df4e81-kube-api-access-fjpps" (OuterVolumeSpecName: "kube-api-access-fjpps") pod "82824644-1af7-4b0c-b220-8702d2df4e81" (UID: "82824644-1af7-4b0c-b220-8702d2df4e81"). InnerVolumeSpecName "kube-api-access-fjpps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:17:54.072506 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.072458    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-wqjwd" podStartSLOduration=1.341574064 podStartE2EDuration="22.07244528s" totalImagesPullingTime="20.730871216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:17:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:17:54.071241754 +0000 UTC m=+419.325031755" watchObservedRunningTime="2026-07-02 23:17:54.07244528 +0000 UTC m=+419.326235309"
Jul 02 23:17:54.091032 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.090972    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wsg8r" podStartSLOduration=12.090952739 podStartE2EDuration="12.090952739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:17:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:17:54.089943972 +0000 UTC m=+419.343733972" watchObservedRunningTime="2026-07-02 23:17:54.090952739 +0000 UTC m=+419.344742740"
Jul 02 23:17:54.092491 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:54.092462    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mxspm\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:54.094660 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:54.094638    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:54.153417 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.153385    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pg2m\" (UniqueName: \"kubernetes.io/projected/10d28a77-5aef-415b-84c9-0582703cefb8-kube-api-access-8pg2m\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:54.153417 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.153417    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjpps\" (UniqueName: \"kubernetes.io/projected/82824644-1af7-4b0c-b220-8702d2df4e81-kube-api-access-fjpps\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:54.153608 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:54.153432    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10d28a77-5aef-415b-84c9-0582703cefb8-extensions-socket-volume\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:54.336262 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:54.336222    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:54.338671 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:54.338642    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mxspm\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:54.340917 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:54.340895    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mxspm\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:54.343072 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:54.343052    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:55.282932 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:55.282896    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4828f\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4828f"
Jul 02 23:17:55.284473 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:55.284454    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10d28a77-5aef-415b-84c9-0582703cefb8" path="/var/lib/kubelet/pods/10d28a77-5aef-415b-84c9-0582703cefb8/volumes"
Jul 02 23:17:55.284762 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:55.284750    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" path="/var/lib/kubelet/pods/82824644-1af7-4b0c-b220-8702d2df4e81/volumes"
Jul 02 23:17:55.285463 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:17:55.285441    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mxspm\" is forbidden: User \"system:node:ip-10-0-140-52.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-52.ec2.internal' and this object" podUID="82824644-1af7-4b0c-b220-8702d2df4e81" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mxspm"
Jul 02 23:17:58.987332 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:58.987295    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"]
Jul 02 23:17:58.987855 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:58.987521    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc" podUID="04a7240a-1b6b-49f9-8911-908bfac52ae1" containerName="manager" containerID="cri-o://44fff592b5e4304fe69d177e557a2c9ac5258086de7f5be3f19419acfee1a652" gracePeriod=10
Jul 02 23:17:58.989520 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:58.989496    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"
Jul 02 23:17:59.230407 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:59.230380    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"
Jul 02 23:17:59.399763 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:59.399678    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmr8k\" (UniqueName: \"kubernetes.io/projected/04a7240a-1b6b-49f9-8911-908bfac52ae1-kube-api-access-zmr8k\") pod \"04a7240a-1b6b-49f9-8911-908bfac52ae1\" (UID: \"04a7240a-1b6b-49f9-8911-908bfac52ae1\") "
Jul 02 23:17:59.399763 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:59.399721    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04a7240a-1b6b-49f9-8911-908bfac52ae1-extensions-socket-volume\") pod \"04a7240a-1b6b-49f9-8911-908bfac52ae1\" (UID: \"04a7240a-1b6b-49f9-8911-908bfac52ae1\") "
Jul 02 23:17:59.400197 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:59.400168    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04a7240a-1b6b-49f9-8911-908bfac52ae1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "04a7240a-1b6b-49f9-8911-908bfac52ae1" (UID: "04a7240a-1b6b-49f9-8911-908bfac52ae1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:17:59.401785 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:59.401763    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04a7240a-1b6b-49f9-8911-908bfac52ae1-kube-api-access-zmr8k" (OuterVolumeSpecName: "kube-api-access-zmr8k") pod "04a7240a-1b6b-49f9-8911-908bfac52ae1" (UID: "04a7240a-1b6b-49f9-8911-908bfac52ae1"). InnerVolumeSpecName "kube-api-access-zmr8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:17:59.500688 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:59.500653    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmr8k\" (UniqueName: \"kubernetes.io/projected/04a7240a-1b6b-49f9-8911-908bfac52ae1-kube-api-access-zmr8k\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:17:59.500688 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:17:59.500685    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04a7240a-1b6b-49f9-8911-908bfac52ae1-extensions-socket-volume\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:18:00.064181 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:00.064143    2580 generic.go:356] "Generic (PLEG): container finished" podID="04a7240a-1b6b-49f9-8911-908bfac52ae1" containerID="44fff592b5e4304fe69d177e557a2c9ac5258086de7f5be3f19419acfee1a652" exitCode=0
Jul 02 23:18:00.064181 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:00.064180    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"
Jul 02 23:18:00.064722 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:00.064216    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc" event={"ID":"04a7240a-1b6b-49f9-8911-908bfac52ae1","Type":"ContainerDied","Data":"44fff592b5e4304fe69d177e557a2c9ac5258086de7f5be3f19419acfee1a652"}
Jul 02 23:18:00.064722 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:00.064274    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc" event={"ID":"04a7240a-1b6b-49f9-8911-908bfac52ae1","Type":"ContainerDied","Data":"b5101315b7bc66a9a78a3b6fcc65936cbf748072849848b92f609032193745a2"}
Jul 02 23:18:00.064722 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:00.064299    2580 scope.go:117] "RemoveContainer" containerID="44fff592b5e4304fe69d177e557a2c9ac5258086de7f5be3f19419acfee1a652"
Jul 02 23:18:00.073052 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:00.073033    2580 scope.go:117] "RemoveContainer" containerID="44fff592b5e4304fe69d177e557a2c9ac5258086de7f5be3f19419acfee1a652"
Jul 02 23:18:00.073384 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:18:00.073363    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44fff592b5e4304fe69d177e557a2c9ac5258086de7f5be3f19419acfee1a652\": container with ID starting with 44fff592b5e4304fe69d177e557a2c9ac5258086de7f5be3f19419acfee1a652 not found: ID does not exist" containerID="44fff592b5e4304fe69d177e557a2c9ac5258086de7f5be3f19419acfee1a652"
Jul 02 23:18:00.073447 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:00.073392    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44fff592b5e4304fe69d177e557a2c9ac5258086de7f5be3f19419acfee1a652"} err="failed to get container status \"44fff592b5e4304fe69d177e557a2c9ac5258086de7f5be3f19419acfee1a652\": rpc error: code = NotFound desc = could not find container \"44fff592b5e4304fe69d177e557a2c9ac5258086de7f5be3f19419acfee1a652\": container with ID starting with 44fff592b5e4304fe69d177e557a2c9ac5258086de7f5be3f19419acfee1a652 not found: ID does not exist"
Jul 02 23:18:00.087002 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:00.086973    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"]
Jul 02 23:18:00.091195 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:00.091152    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zw9cc"]
Jul 02 23:18:01.282360 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:01.282329    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04a7240a-1b6b-49f9-8911-908bfac52ae1" path="/var/lib/kubelet/pods/04a7240a-1b6b-49f9-8911-908bfac52ae1/volumes"
Jul 02 23:18:05.041364 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:05.041337    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wsg8r"
Jul 02 23:18:15.380073 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.380038    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"]
Jul 02 23:18:15.380501 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.380364    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04a7240a-1b6b-49f9-8911-908bfac52ae1" containerName="manager"
Jul 02 23:18:15.380501 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.380376    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="04a7240a-1b6b-49f9-8911-908bfac52ae1" containerName="manager"
Jul 02 23:18:15.380501 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.380431    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="04a7240a-1b6b-49f9-8911-908bfac52ae1" containerName="manager"
Jul 02 23:18:15.550717 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.550683    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"]
Jul 02 23:18:15.550958 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.550808    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.553574 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.553552    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-6bhcv\""
Jul 02 23:18:15.553934 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.553920    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 02 23:18:15.620363 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.620327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.620531 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.620376    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.620531 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.620446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.620531 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.620494    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.620531 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.620526    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.620677 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.620546    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvw6s\" (UniqueName: \"kubernetes.io/projected/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-kube-api-access-wvw6s\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.620677 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.620575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.620677 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.620603    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.620677 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.620621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.721313 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.721282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.721313 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.721321    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvw6s\" (UniqueName: \"kubernetes.io/projected/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-kube-api-access-wvw6s\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.721546 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.721344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.721546 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.721470    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.721546 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.721507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.721663 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.721555    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.721663 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.721585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.721663 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.721627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.721804 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.721677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.721804 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.721765    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.721916 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.721881    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.722058 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.722037    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.722141 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.722058    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.722328 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.722306    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.723933 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.723907    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.724095 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.724079    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.729873 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.729845    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.730503 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.730451    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvw6s\" (UniqueName: \"kubernetes.io/projected/23482fd4-bf4d-4bf7-a5ae-ab019df7f807-kube-api-access-wvw6s\") pod \"maas-default-gateway-openshift-default-56654dcf7-c9ld8\" (UID: \"23482fd4-bf4d-4bf7-a5ae-ab019df7f807\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:15.861597 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:15.861558    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:16.027905 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:16.027879    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"]
Jul 02 23:18:16.030525 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:18:16.030496    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23482fd4_bf4d_4bf7_a5ae_ab019df7f807.slice/crio-bad210c00e5bcfe78e95de40bb8499a1254ea74fdf113b41c1a39a4e6f579deb WatchSource:0}: Error finding container bad210c00e5bcfe78e95de40bb8499a1254ea74fdf113b41c1a39a4e6f579deb: Status 404 returned error can't find the container with id bad210c00e5bcfe78e95de40bb8499a1254ea74fdf113b41c1a39a4e6f579deb
Jul 02 23:18:16.140651 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:16.140614    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8" event={"ID":"23482fd4-bf4d-4bf7-a5ae-ab019df7f807","Type":"ContainerStarted","Data":"bad210c00e5bcfe78e95de40bb8499a1254ea74fdf113b41c1a39a4e6f579deb"}
Jul 02 23:18:18.918990 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:18.918950    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:18:18.919263 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:18.919029    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:18:18.919263 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:18.919060    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:18:19.158577 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:19.158536    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8" event={"ID":"23482fd4-bf4d-4bf7-a5ae-ab019df7f807","Type":"ContainerStarted","Data":"104f19bb692f3726f87e5c603a4be1c1198354f78a017905f61bd53f3b5b0c53"}
Jul 02 23:18:19.186101 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:19.186032    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8" podStartSLOduration=1.3002654310000001 podStartE2EDuration="4.186010326s" totalImagesPullingTime="2.885744895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:18:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:18:19.184239286 +0000 UTC m=+444.438029286" watchObservedRunningTime="2026-07-02 23:18:19.186010326 +0000 UTC m=+444.439800318"
Jul 02 23:18:19.861914 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:19.861866    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:19.866953 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:19.866922    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:20.164637 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.164562    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:20.165568 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.165545    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:18:20.239341 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.239298    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"]
Jul 02 23:18:20.644876 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.644841    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"]
Jul 02 23:18:20.644876 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.644878    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"]
Jul 02 23:18:20.645107 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.644895    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:18:20.645107 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.644910    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"
Jul 02 23:18:20.647709 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.647681    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 02 23:18:20.694371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.694324    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:18:20.694371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.694368    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:18:20.694574 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.694469    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-5czdx"
Jul 02 23:18:20.766749 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.766708    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrd6k\" (UniqueName: \"kubernetes.io/projected/8311e635-8c00-435c-af36-7b6a62cbba5d-kube-api-access-jrd6k\") pod \"limitador-limitador-54d6d988d5-5czdx\" (UID: \"8311e635-8c00-435c-af36-7b6a62cbba5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5czdx"
Jul 02 23:18:20.766977 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.766777    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrr9r\" (UniqueName: \"kubernetes.io/projected/ad7410be-5a3e-4e07-8a56-419a5f1e27df-kube-api-access-jrr9r\") pod \"limitador-limitador-64c94fd64f-rj6c9\" (UID: \"ad7410be-5a3e-4e07-8a56-419a5f1e27df\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"
Jul 02 23:18:20.767159 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.766985    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8311e635-8c00-435c-af36-7b6a62cbba5d-config-file\") pod \"limitador-limitador-54d6d988d5-5czdx\" (UID: \"8311e635-8c00-435c-af36-7b6a62cbba5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5czdx"
Jul 02 23:18:20.767159 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.767074    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad7410be-5a3e-4e07-8a56-419a5f1e27df-config-file\") pod \"limitador-limitador-64c94fd64f-rj6c9\" (UID: \"ad7410be-5a3e-4e07-8a56-419a5f1e27df\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"
Jul 02 23:18:20.867785 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.867747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrd6k\" (UniqueName: \"kubernetes.io/projected/8311e635-8c00-435c-af36-7b6a62cbba5d-kube-api-access-jrd6k\") pod \"limitador-limitador-54d6d988d5-5czdx\" (UID: \"8311e635-8c00-435c-af36-7b6a62cbba5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5czdx"
Jul 02 23:18:20.867785 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.867793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrr9r\" (UniqueName: \"kubernetes.io/projected/ad7410be-5a3e-4e07-8a56-419a5f1e27df-kube-api-access-jrr9r\") pod \"limitador-limitador-64c94fd64f-rj6c9\" (UID: \"ad7410be-5a3e-4e07-8a56-419a5f1e27df\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"
Jul 02 23:18:20.868059 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.867827    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8311e635-8c00-435c-af36-7b6a62cbba5d-config-file\") pod \"limitador-limitador-54d6d988d5-5czdx\" (UID: \"8311e635-8c00-435c-af36-7b6a62cbba5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5czdx"
Jul 02 23:18:20.868059 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.867846    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad7410be-5a3e-4e07-8a56-419a5f1e27df-config-file\") pod \"limitador-limitador-64c94fd64f-rj6c9\" (UID: \"ad7410be-5a3e-4e07-8a56-419a5f1e27df\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"
Jul 02 23:18:20.868507 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.868490    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad7410be-5a3e-4e07-8a56-419a5f1e27df-config-file\") pod \"limitador-limitador-64c94fd64f-rj6c9\" (UID: \"ad7410be-5a3e-4e07-8a56-419a5f1e27df\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"
Jul 02 23:18:20.868570 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.868547    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8311e635-8c00-435c-af36-7b6a62cbba5d-config-file\") pod \"limitador-limitador-54d6d988d5-5czdx\" (UID: \"8311e635-8c00-435c-af36-7b6a62cbba5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5czdx"
Jul 02 23:18:20.876082 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.876054    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrd6k\" (UniqueName: \"kubernetes.io/projected/8311e635-8c00-435c-af36-7b6a62cbba5d-kube-api-access-jrd6k\") pod \"limitador-limitador-54d6d988d5-5czdx\" (UID: \"8311e635-8c00-435c-af36-7b6a62cbba5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5czdx"
Jul 02 23:18:20.876228 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.876053    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrr9r\" (UniqueName: \"kubernetes.io/projected/ad7410be-5a3e-4e07-8a56-419a5f1e27df-kube-api-access-jrr9r\") pod \"limitador-limitador-64c94fd64f-rj6c9\" (UID: \"ad7410be-5a3e-4e07-8a56-419a5f1e27df\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"
Jul 02 23:18:20.956063 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:20.956023    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"
Jul 02 23:18:21.005017 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:21.004974    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-5czdx"
Jul 02 23:18:21.126880 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:21.126848    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"]
Jul 02 23:18:21.130769 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:18:21.130727    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad7410be_5a3e_4e07_8a56_419a5f1e27df.slice/crio-607142746f89407890fd3ad941fe7ee6f75d0150f2fd67d445e3f780b66c5f7d WatchSource:0}: Error finding container 607142746f89407890fd3ad941fe7ee6f75d0150f2fd67d445e3f780b66c5f7d: Status 404 returned error can't find the container with id 607142746f89407890fd3ad941fe7ee6f75d0150f2fd67d445e3f780b66c5f7d
Jul 02 23:18:21.165413 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:21.165383    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:18:21.169101 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:18:21.169071    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8311e635_8c00_435c_af36_7b6a62cbba5d.slice/crio-9f4e54f6167a5902c4aa92e9f5e203603a5b06e6f386e1fdf6e1e589b24ef93d WatchSource:0}: Error finding container 9f4e54f6167a5902c4aa92e9f5e203603a5b06e6f386e1fdf6e1e589b24ef93d: Status 404 returned error can't find the container with id 9f4e54f6167a5902c4aa92e9f5e203603a5b06e6f386e1fdf6e1e589b24ef93d
Jul 02 23:18:21.173764 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:21.173630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9" event={"ID":"ad7410be-5a3e-4e07-8a56-419a5f1e27df","Type":"ContainerStarted","Data":"607142746f89407890fd3ad941fe7ee6f75d0150f2fd67d445e3f780b66c5f7d"}
Jul 02 23:18:22.179390 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:22.179354    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-5czdx" event={"ID":"8311e635-8c00-435c-af36-7b6a62cbba5d","Type":"ContainerStarted","Data":"9f4e54f6167a5902c4aa92e9f5e203603a5b06e6f386e1fdf6e1e589b24ef93d"}
Jul 02 23:18:26.205643 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:26.205596    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9" event={"ID":"ad7410be-5a3e-4e07-8a56-419a5f1e27df","Type":"ContainerStarted","Data":"7b0ab003e3fbfc0d8466b80b5834c827abc06e223b8cb59809bb83e8f016c9a3"}
Jul 02 23:18:26.206080 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:26.205670    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"
Jul 02 23:18:26.207159 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:26.207133    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-5czdx" event={"ID":"8311e635-8c00-435c-af36-7b6a62cbba5d","Type":"ContainerStarted","Data":"94bf0791d9ed7003b3cbee1642819babc6700cddf8a3be0672685f8acdc0b48d"}
Jul 02 23:18:26.207297 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:26.207192    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-5czdx"
Jul 02 23:18:26.223363 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:26.223310    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9" podStartSLOduration=2.212335299 podStartE2EDuration="6.223294725s" totalImagesPullingTime="4.010959426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:18:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:18:26.222423921 +0000 UTC m=+451.476213922" watchObservedRunningTime="2026-07-02 23:18:26.223294725 +0000 UTC m=+451.477084723"
Jul 02 23:18:26.240810 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:26.240752    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-5czdx" podStartSLOduration=2.283929615 podStartE2EDuration="6.240733825s" totalImagesPullingTime="3.95680421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:18:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:18:26.239115034 +0000 UTC m=+451.492929984" watchObservedRunningTime="2026-07-02 23:18:26.240733825 +0000 UTC m=+451.494523824"
Jul 02 23:18:37.213243 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:37.213210    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"
Jul 02 23:18:37.213828 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:37.213284    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-5czdx"
Jul 02 23:18:37.274854 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:37.274819    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"]
Jul 02 23:18:37.275071 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:37.275044    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9" podUID="ad7410be-5a3e-4e07-8a56-419a5f1e27df" containerName="limitador" containerID="cri-o://7b0ab003e3fbfc0d8466b80b5834c827abc06e223b8cb59809bb83e8f016c9a3" gracePeriod=30
Jul 02 23:18:37.823650 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:37.823625    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"
Jul 02 23:18:37.913292 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:37.913208    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad7410be-5a3e-4e07-8a56-419a5f1e27df-config-file\") pod \"ad7410be-5a3e-4e07-8a56-419a5f1e27df\" (UID: \"ad7410be-5a3e-4e07-8a56-419a5f1e27df\") "
Jul 02 23:18:37.913292 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:37.913260    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrr9r\" (UniqueName: \"kubernetes.io/projected/ad7410be-5a3e-4e07-8a56-419a5f1e27df-kube-api-access-jrr9r\") pod \"ad7410be-5a3e-4e07-8a56-419a5f1e27df\" (UID: \"ad7410be-5a3e-4e07-8a56-419a5f1e27df\") "
Jul 02 23:18:37.913650 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:37.913623    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad7410be-5a3e-4e07-8a56-419a5f1e27df-config-file" (OuterVolumeSpecName: "config-file") pod "ad7410be-5a3e-4e07-8a56-419a5f1e27df" (UID: "ad7410be-5a3e-4e07-8a56-419a5f1e27df"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 23:18:37.915374 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:37.915336    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad7410be-5a3e-4e07-8a56-419a5f1e27df-kube-api-access-jrr9r" (OuterVolumeSpecName: "kube-api-access-jrr9r") pod "ad7410be-5a3e-4e07-8a56-419a5f1e27df" (UID: "ad7410be-5a3e-4e07-8a56-419a5f1e27df"). InnerVolumeSpecName "kube-api-access-jrr9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:18:38.014660 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:38.014627    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrr9r\" (UniqueName: \"kubernetes.io/projected/ad7410be-5a3e-4e07-8a56-419a5f1e27df-kube-api-access-jrr9r\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:18:38.014660 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:38.014654    2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad7410be-5a3e-4e07-8a56-419a5f1e27df-config-file\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:18:38.266003 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:38.265969    2580 generic.go:356] "Generic (PLEG): container finished" podID="ad7410be-5a3e-4e07-8a56-419a5f1e27df" containerID="7b0ab003e3fbfc0d8466b80b5834c827abc06e223b8cb59809bb83e8f016c9a3" exitCode=0
Jul 02 23:18:38.266476 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:38.266017    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"
Jul 02 23:18:38.266476 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:38.266030    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9" event={"ID":"ad7410be-5a3e-4e07-8a56-419a5f1e27df","Type":"ContainerDied","Data":"7b0ab003e3fbfc0d8466b80b5834c827abc06e223b8cb59809bb83e8f016c9a3"}
Jul 02 23:18:38.266476 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:38.266071    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-rj6c9" event={"ID":"ad7410be-5a3e-4e07-8a56-419a5f1e27df","Type":"ContainerDied","Data":"607142746f89407890fd3ad941fe7ee6f75d0150f2fd67d445e3f780b66c5f7d"}
Jul 02 23:18:38.266476 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:38.266093    2580 scope.go:117] "RemoveContainer" containerID="7b0ab003e3fbfc0d8466b80b5834c827abc06e223b8cb59809bb83e8f016c9a3"
Jul 02 23:18:38.274891 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:38.274872    2580 scope.go:117] "RemoveContainer" containerID="7b0ab003e3fbfc0d8466b80b5834c827abc06e223b8cb59809bb83e8f016c9a3"
Jul 02 23:18:38.275175 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:18:38.275156    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b0ab003e3fbfc0d8466b80b5834c827abc06e223b8cb59809bb83e8f016c9a3\": container with ID starting with 7b0ab003e3fbfc0d8466b80b5834c827abc06e223b8cb59809bb83e8f016c9a3 not found: ID does not exist" containerID="7b0ab003e3fbfc0d8466b80b5834c827abc06e223b8cb59809bb83e8f016c9a3"
Jul 02 23:18:38.275246 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:38.275184    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b0ab003e3fbfc0d8466b80b5834c827abc06e223b8cb59809bb83e8f016c9a3"} err="failed to get container status \"7b0ab003e3fbfc0d8466b80b5834c827abc06e223b8cb59809bb83e8f016c9a3\": rpc error: code = NotFound desc = could not find container \"7b0ab003e3fbfc0d8466b80b5834c827abc06e223b8cb59809bb83e8f016c9a3\": container with ID starting with 7b0ab003e3fbfc0d8466b80b5834c827abc06e223b8cb59809bb83e8f016c9a3 not found: ID does not exist"
Jul 02 23:18:38.289043 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:38.289010    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"]
Jul 02 23:18:38.294928 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:38.294903    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-rj6c9"]
Jul 02 23:18:39.282432 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:39.282399    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad7410be-5a3e-4e07-8a56-419a5f1e27df" path="/var/lib/kubelet/pods/ad7410be-5a3e-4e07-8a56-419a5f1e27df/volumes"
Jul 02 23:18:41.824811 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:41.824777    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-qj4zs"]
Jul 02 23:18:41.825223 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:41.825129    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad7410be-5a3e-4e07-8a56-419a5f1e27df" containerName="limitador"
Jul 02 23:18:41.825223 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:41.825145    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad7410be-5a3e-4e07-8a56-419a5f1e27df" containerName="limitador"
Jul 02 23:18:41.825223 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:41.825218    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad7410be-5a3e-4e07-8a56-419a5f1e27df" containerName="limitador"
Jul 02 23:18:41.829758 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:41.829729    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-qj4zs"
Jul 02 23:18:41.832678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:41.832657    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
Jul 02 23:18:41.832799 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:41.832759    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kb6d6\""
Jul 02 23:18:41.837690 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:41.837664    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-qj4zs"]
Jul 02 23:18:41.949906 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:41.949868    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7b374e4f-26ff-4f69-9ed0-a5f607a28270-data\") pod \"postgres-69fb4594cf-qj4zs\" (UID: \"7b374e4f-26ff-4f69-9ed0-a5f607a28270\") " pod="opendatahub/postgres-69fb4594cf-qj4zs"
Jul 02 23:18:41.950101 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:41.949936    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmgkk\" (UniqueName: \"kubernetes.io/projected/7b374e4f-26ff-4f69-9ed0-a5f607a28270-kube-api-access-nmgkk\") pod \"postgres-69fb4594cf-qj4zs\" (UID: \"7b374e4f-26ff-4f69-9ed0-a5f607a28270\") " pod="opendatahub/postgres-69fb4594cf-qj4zs"
Jul 02 23:18:42.051214 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:42.051172    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7b374e4f-26ff-4f69-9ed0-a5f607a28270-data\") pod \"postgres-69fb4594cf-qj4zs\" (UID: \"7b374e4f-26ff-4f69-9ed0-a5f607a28270\") " pod="opendatahub/postgres-69fb4594cf-qj4zs"
Jul 02 23:18:42.051347 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:42.051240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmgkk\" (UniqueName: \"kubernetes.io/projected/7b374e4f-26ff-4f69-9ed0-a5f607a28270-kube-api-access-nmgkk\") pod \"postgres-69fb4594cf-qj4zs\" (UID: \"7b374e4f-26ff-4f69-9ed0-a5f607a28270\") " pod="opendatahub/postgres-69fb4594cf-qj4zs"
Jul 02 23:18:42.051576 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:42.051554    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7b374e4f-26ff-4f69-9ed0-a5f607a28270-data\") pod \"postgres-69fb4594cf-qj4zs\" (UID: \"7b374e4f-26ff-4f69-9ed0-a5f607a28270\") " pod="opendatahub/postgres-69fb4594cf-qj4zs"
Jul 02 23:18:42.060292 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:42.060264    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmgkk\" (UniqueName: \"kubernetes.io/projected/7b374e4f-26ff-4f69-9ed0-a5f607a28270-kube-api-access-nmgkk\") pod \"postgres-69fb4594cf-qj4zs\" (UID: \"7b374e4f-26ff-4f69-9ed0-a5f607a28270\") " pod="opendatahub/postgres-69fb4594cf-qj4zs"
Jul 02 23:18:42.142275 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:42.142188    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-qj4zs"
Jul 02 23:18:42.275530 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:42.275500    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-qj4zs"]
Jul 02 23:18:42.277543 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:18:42.277517    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b374e4f_26ff_4f69_9ed0_a5f607a28270.slice/crio-c1d7ec53bc6914cacf60b1b6fc7a411668249cecf78c75fb3844a1862fd9015a WatchSource:0}: Error finding container c1d7ec53bc6914cacf60b1b6fc7a411668249cecf78c75fb3844a1862fd9015a: Status 404 returned error can't find the container with id c1d7ec53bc6914cacf60b1b6fc7a411668249cecf78c75fb3844a1862fd9015a
Jul 02 23:18:42.287480 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:42.287447    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-qj4zs" event={"ID":"7b374e4f-26ff-4f69-9ed0-a5f607a28270","Type":"ContainerStarted","Data":"c1d7ec53bc6914cacf60b1b6fc7a411668249cecf78c75fb3844a1862fd9015a"}
Jul 02 23:18:51.333172 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:51.333111    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-qj4zs" event={"ID":"7b374e4f-26ff-4f69-9ed0-a5f607a28270","Type":"ContainerStarted","Data":"fbb46e9718897c4d7b42262cd09a1a178c33f5e1cf4e310cd026d38463d9c654"}
Jul 02 23:18:51.333560 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:51.333260    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-qj4zs"
Jul 02 23:18:51.351525 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:51.351478    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-qj4zs" podStartSLOduration=2.2696221100000002 podStartE2EDuration="10.351463484s" totalImagesPullingTime="8.081841374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:18:51.348988128 +0000 UTC m=+476.602778139" watchObservedRunningTime="2026-07-02 23:18:51.351463484 +0000 UTC m=+476.605253483"
Jul 02 23:18:57.373738 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:18:57.373704    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-qj4zs"
Jul 02 23:19:40.132560 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:40.132527    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-rbl9d"]
Jul 02 23:19:40.150963 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:40.150934    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-rbl9d"]
Jul 02 23:19:40.151135 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:40.151033    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-rbl9d"
Jul 02 23:19:40.154191 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:40.154167    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-k6j8t\""
Jul 02 23:19:40.154191 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:40.154183    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 02 23:19:40.238181 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:40.238098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvkxc\" (UniqueName: \"kubernetes.io/projected/acc52d0a-7366-468b-8a0a-8b1240a7facd-kube-api-access-zvkxc\") pod \"payload-processing-58f86c7798-rbl9d\" (UID: \"acc52d0a-7366-468b-8a0a-8b1240a7facd\") " pod="openshift-ingress/payload-processing-58f86c7798-rbl9d"
Jul 02 23:19:40.238378 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:40.238273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acc52d0a-7366-468b-8a0a-8b1240a7facd-plugins-config-volume\") pod \"payload-processing-58f86c7798-rbl9d\" (UID: \"acc52d0a-7366-468b-8a0a-8b1240a7facd\") " pod="openshift-ingress/payload-processing-58f86c7798-rbl9d"
Jul 02 23:19:40.272747 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:40.272712    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:19:40.339254 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:40.339218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvkxc\" (UniqueName: \"kubernetes.io/projected/acc52d0a-7366-468b-8a0a-8b1240a7facd-kube-api-access-zvkxc\") pod \"payload-processing-58f86c7798-rbl9d\" (UID: \"acc52d0a-7366-468b-8a0a-8b1240a7facd\") " pod="openshift-ingress/payload-processing-58f86c7798-rbl9d"
Jul 02 23:19:40.339477 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:40.339454    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acc52d0a-7366-468b-8a0a-8b1240a7facd-plugins-config-volume\") pod \"payload-processing-58f86c7798-rbl9d\" (UID: \"acc52d0a-7366-468b-8a0a-8b1240a7facd\") " pod="openshift-ingress/payload-processing-58f86c7798-rbl9d"
Jul 02 23:19:40.340101 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:40.340074    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acc52d0a-7366-468b-8a0a-8b1240a7facd-plugins-config-volume\") pod \"payload-processing-58f86c7798-rbl9d\" (UID: \"acc52d0a-7366-468b-8a0a-8b1240a7facd\") " pod="openshift-ingress/payload-processing-58f86c7798-rbl9d"
Jul 02 23:19:40.349776 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:40.349738    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvkxc\" (UniqueName: \"kubernetes.io/projected/acc52d0a-7366-468b-8a0a-8b1240a7facd-kube-api-access-zvkxc\") pod \"payload-processing-58f86c7798-rbl9d\" (UID: \"acc52d0a-7366-468b-8a0a-8b1240a7facd\") " pod="openshift-ingress/payload-processing-58f86c7798-rbl9d"
Jul 02 23:19:40.461903 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:40.461854    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-rbl9d"
Jul 02 23:19:40.597823 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:40.597798    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-rbl9d"]
Jul 02 23:19:40.600593 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:19:40.600557    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacc52d0a_7366_468b_8a0a_8b1240a7facd.slice/crio-f693511acd192d3c305c948c2fb856c808e4669f709c1e70172da581a1c6417b WatchSource:0}: Error finding container f693511acd192d3c305c948c2fb856c808e4669f709c1e70172da581a1c6417b: Status 404 returned error can't find the container with id f693511acd192d3c305c948c2fb856c808e4669f709c1e70172da581a1c6417b
Jul 02 23:19:41.570673 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:41.570643    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-rbl9d" event={"ID":"acc52d0a-7366-468b-8a0a-8b1240a7facd","Type":"ContainerStarted","Data":"f693511acd192d3c305c948c2fb856c808e4669f709c1e70172da581a1c6417b"}
Jul 02 23:19:42.576664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:42.576626    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-rbl9d" event={"ID":"acc52d0a-7366-468b-8a0a-8b1240a7facd","Type":"ContainerStarted","Data":"9b589d94119d69860808b6baa29018e45422c15c84b6231f9acb75d8f269a84b"}
Jul 02 23:19:42.577092 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:42.576810    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-rbl9d"
Jul 02 23:19:42.594740 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:42.594692    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-rbl9d" podStartSLOduration=1.313784187 podStartE2EDuration="2.594676319s" totalImagesPullingTime="1.280892132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:19:42.593770836 +0000 UTC m=+527.847560846" watchObservedRunningTime="2026-07-02 23:19:42.594676319 +0000 UTC m=+527.848466317"
Jul 02 23:19:48.608030 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:48.607989    2580 generic.go:356] "Generic (PLEG): container finished" podID="23482fd4-bf4d-4bf7-a5ae-ab019df7f807" containerID="104f19bb692f3726f87e5c603a4be1c1198354f78a017905f61bd53f3b5b0c53" exitCode=0
Jul 02 23:19:48.608596 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:48.608041    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8" event={"ID":"23482fd4-bf4d-4bf7-a5ae-ab019df7f807","Type":"ContainerDied","Data":"104f19bb692f3726f87e5c603a4be1c1198354f78a017905f61bd53f3b5b0c53"}
Jul 02 23:19:48.608596 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:48.608585    2580 scope.go:117] "RemoveContainer" containerID="104f19bb692f3726f87e5c603a4be1c1198354f78a017905f61bd53f3b5b0c53"
Jul 02 23:19:48.609602 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:48.609572    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:19:48.609664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:48.609643    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:19:48.609701 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:48.609677    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:19:49.615094 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:49.615059    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8" event={"ID":"23482fd4-bf4d-4bf7-a5ae-ab019df7f807","Type":"ContainerStarted","Data":"df7feca69a31e5c1b388f0e3bfd00ad662e3e341d9aeb6acaaf51615b4066488"}
Jul 02 23:19:49.862284 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:49.862243    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:19:49.863389 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:49.863361    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-c9ld8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 02 23:19:49.863518 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:49.863408    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8" podUID="23482fd4-bf4d-4bf7-a5ae-ab019df7f807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 02 23:19:50.164804 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:50.164761    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:19:50.862325 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:50.862287    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-c9ld8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 02 23:19:50.862706 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:50.862347    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8" podUID="23482fd4-bf4d-4bf7-a5ae-ab019df7f807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 02 23:19:51.862452 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:51.862414    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-c9ld8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 02 23:19:51.862857 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:51.862470    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8" podUID="23482fd4-bf4d-4bf7-a5ae-ab019df7f807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 02 23:19:52.862807 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:52.862767    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-c9ld8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 02 23:19:52.863229 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:52.862824    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8" podUID="23482fd4-bf4d-4bf7-a5ae-ab019df7f807" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 02 23:19:53.583296 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:53.583266    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-rbl9d"
Jul 02 23:19:53.866180 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:53.866080    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:19:53.867358 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:19:53.867338    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-c9ld8"
Jul 02 23:20:00.796820 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:00.796782    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:20:24.784199 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:24.784144    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:20:45.790706 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:45.790672    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"]
Jul 02 23:20:45.794166 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:45.794145    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:45.798350 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:45.798326    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 02 23:20:45.798480 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:45.798445    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 02 23:20:45.798480 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:45.798463    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 02 23:20:45.798602 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:45.798584    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8wqs2\""
Jul 02 23:20:45.806556 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:45.806534    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"]
Jul 02 23:20:45.910271 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:45.910234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a10b5aa4-5198-42de-bda1-8054dd5caefa-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:45.910271 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:45.910288    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a10b5aa4-5198-42de-bda1-8054dd5caefa-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:45.910553 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:45.910304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a10b5aa4-5198-42de-bda1-8054dd5caefa-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:45.910553 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:45.910405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a10b5aa4-5198-42de-bda1-8054dd5caefa-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:45.910553 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:45.910438    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a10b5aa4-5198-42de-bda1-8054dd5caefa-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:45.910553 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:45.910487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k98zp\" (UniqueName: \"kubernetes.io/projected/a10b5aa4-5198-42de-bda1-8054dd5caefa-kube-api-access-k98zp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:46.011458 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.011413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a10b5aa4-5198-42de-bda1-8054dd5caefa-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:46.011669 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.011482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a10b5aa4-5198-42de-bda1-8054dd5caefa-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:46.011669 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.011502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a10b5aa4-5198-42de-bda1-8054dd5caefa-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:46.011669 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.011545    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a10b5aa4-5198-42de-bda1-8054dd5caefa-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:46.011669 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.011562    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a10b5aa4-5198-42de-bda1-8054dd5caefa-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:46.011669 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.011598    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k98zp\" (UniqueName: \"kubernetes.io/projected/a10b5aa4-5198-42de-bda1-8054dd5caefa-kube-api-access-k98zp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:46.011967 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.011942    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a10b5aa4-5198-42de-bda1-8054dd5caefa-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:46.012017 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.011972    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a10b5aa4-5198-42de-bda1-8054dd5caefa-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:46.012017 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.011995    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a10b5aa4-5198-42de-bda1-8054dd5caefa-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:46.013762 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.013735    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a10b5aa4-5198-42de-bda1-8054dd5caefa-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:46.013936 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.013916    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a10b5aa4-5198-42de-bda1-8054dd5caefa-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:46.022863 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.022829    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k98zp\" (UniqueName: \"kubernetes.io/projected/a10b5aa4-5198-42de-bda1-8054dd5caefa-kube-api-access-k98zp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wvdzz\" (UID: \"a10b5aa4-5198-42de-bda1-8054dd5caefa\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:46.105380 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.105282    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:46.240408 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.240379    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"]
Jul 02 23:20:46.242678 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:20:46.242649    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda10b5aa4_5198_42de_bda1_8054dd5caefa.slice/crio-7fba83503a60b40a0f5b7b28d165b7765dc1c40508dcd212d8f2af8c72218403 WatchSource:0}: Error finding container 7fba83503a60b40a0f5b7b28d165b7765dc1c40508dcd212d8f2af8c72218403: Status 404 returned error can't find the container with id 7fba83503a60b40a0f5b7b28d165b7765dc1c40508dcd212d8f2af8c72218403
Jul 02 23:20:46.882666 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.882633    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:20:46.891374 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:46.891346    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz" event={"ID":"a10b5aa4-5198-42de-bda1-8054dd5caefa","Type":"ContainerStarted","Data":"7fba83503a60b40a0f5b7b28d165b7765dc1c40508dcd212d8f2af8c72218403"}
Jul 02 23:20:47.897445 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:47.897405    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz" event={"ID":"a10b5aa4-5198-42de-bda1-8054dd5caefa","Type":"ContainerStarted","Data":"255fb18cb97e06cd76072f36b9616c0fe3d7ed179e64a2d380f9410ea4138d7b"}
Jul 02 23:20:47.917983 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:47.917925    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz" podStartSLOduration=1.8859496820000001 podStartE2EDuration="2.917908925s" totalImagesPullingTime="1.031959243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:20:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:20:47.916416028 +0000 UTC m=+593.170206028" watchObservedRunningTime="2026-07-02 23:20:47.917908925 +0000 UTC m=+593.171698923"
Jul 02 23:20:56.106097 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:56.106051    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:56.106631 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:56.106151    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:56.118693 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:56.118661    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:56.954029 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:56.953998    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wvdzz"
Jul 02 23:20:58.979888 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:20:58.979852    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:21:03.083654 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.083620    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"]
Jul 02 23:21:03.090005 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.089982    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.092666 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.092635    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 02 23:21:03.098797 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.098757    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"]
Jul 02 23:21:03.162100 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.162054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.162100 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.162101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.162357 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.162153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.162357 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.162176    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.162357 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.162227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.162357 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.162260    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klrqj\" (UniqueName: \"kubernetes.io/projected/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-kube-api-access-klrqj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.263670 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.263632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.263670 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.263671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.263881 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.263690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.263881 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.263820    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.263881 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.263874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.263984 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.263928    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klrqj\" (UniqueName: \"kubernetes.io/projected/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-kube-api-access-klrqj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.264075 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.264057    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.264147 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.264098    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.264254 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.264236    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.265976 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.265953    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.266274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.266256    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.276686 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.276649    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klrqj\" (UniqueName: \"kubernetes.io/projected/1d9a71e6-6de3-4d94-b255-2d0f5c30ead5-kube-api-access-klrqj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7ttdh\" (UID: \"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.402351 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.402258    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:03.540874 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.540853    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"]
Jul 02 23:21:03.542904 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:21:03.542875    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d9a71e6_6de3_4d94_b255_2d0f5c30ead5.slice/crio-633f30c822c1f81041e726ed308fb50f1be1fbaa884fb3fb52cff23987a278b5 WatchSource:0}: Error finding container 633f30c822c1f81041e726ed308fb50f1be1fbaa884fb3fb52cff23987a278b5: Status 404 returned error can't find the container with id 633f30c822c1f81041e726ed308fb50f1be1fbaa884fb3fb52cff23987a278b5
Jul 02 23:21:03.544349 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.544333    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 02 23:21:03.978515 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.978409    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh" event={"ID":"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5","Type":"ContainerStarted","Data":"fdd9502e6c6cdbc3ac6633a0d37902e08535a9da45e4a4b3801cd8f7a2c07dd3"}
Jul 02 23:21:03.978515 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.978470    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh" event={"ID":"1d9a71e6-6de3-4d94-b255-2d0f5c30ead5","Type":"ContainerStarted","Data":"633f30c822c1f81041e726ed308fb50f1be1fbaa884fb3fb52cff23987a278b5"}
Jul 02 23:21:03.998227 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:03.998167    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh" podStartSLOduration=0.821729583 podStartE2EDuration="998.150937ms" totalImagesPullingTime="176.421354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:21:03.995697263 +0000 UTC m=+609.249487262" watchObservedRunningTime="2026-07-02 23:21:03.998150937 +0000 UTC m=+609.251940935"
Jul 02 23:21:04.773114 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:04.773076    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:21:07.284938 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.284907    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"]
Jul 02 23:21:07.289523 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.289502    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.292566 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.292545    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 02 23:21:07.298024 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.298000    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"]
Jul 02 23:21:07.403269 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.403220    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/144b0851-31ea-4c6d-b3cb-e7256b934511-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.403269 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.403274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/144b0851-31ea-4c6d-b3cb-e7256b934511-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.403533 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.403360    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsfzk\" (UniqueName: \"kubernetes.io/projected/144b0851-31ea-4c6d-b3cb-e7256b934511-kube-api-access-jsfzk\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.403533 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.403434    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/144b0851-31ea-4c6d-b3cb-e7256b934511-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.403533 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.403462    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/144b0851-31ea-4c6d-b3cb-e7256b934511-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.403533 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.403486    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/144b0851-31ea-4c6d-b3cb-e7256b934511-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.504828 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.504785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/144b0851-31ea-4c6d-b3cb-e7256b934511-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.504828 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.504830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/144b0851-31ea-4c6d-b3cb-e7256b934511-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.505039 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.504861    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsfzk\" (UniqueName: \"kubernetes.io/projected/144b0851-31ea-4c6d-b3cb-e7256b934511-kube-api-access-jsfzk\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.505039 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.504988    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/144b0851-31ea-4c6d-b3cb-e7256b934511-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.505259 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.505039    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/144b0851-31ea-4c6d-b3cb-e7256b934511-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.505259 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.505066    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/144b0851-31ea-4c6d-b3cb-e7256b934511-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.505431 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.505413    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/144b0851-31ea-4c6d-b3cb-e7256b934511-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.505519 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.505499    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/144b0851-31ea-4c6d-b3cb-e7256b934511-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.505582 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.505541    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/144b0851-31ea-4c6d-b3cb-e7256b934511-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.507129 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.507097    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/144b0851-31ea-4c6d-b3cb-e7256b934511-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.507534 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.507513    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/144b0851-31ea-4c6d-b3cb-e7256b934511-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.513938 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.513913    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsfzk\" (UniqueName: \"kubernetes.io/projected/144b0851-31ea-4c6d-b3cb-e7256b934511-kube-api-access-jsfzk\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp\" (UID: \"144b0851-31ea-4c6d-b3cb-e7256b934511\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.600404 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.600299    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:07.737149 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:07.737110    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"]
Jul 02 23:21:07.739017 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:21:07.738980    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod144b0851_31ea_4c6d_b3cb_e7256b934511.slice/crio-307f7f37fe22267ad86afdb024995672e3c1a1acfb67cbb726e231aba95da2be WatchSource:0}: Error finding container 307f7f37fe22267ad86afdb024995672e3c1a1acfb67cbb726e231aba95da2be: Status 404 returned error can't find the container with id 307f7f37fe22267ad86afdb024995672e3c1a1acfb67cbb726e231aba95da2be
Jul 02 23:21:08.002232 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:08.002198    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp" event={"ID":"144b0851-31ea-4c6d-b3cb-e7256b934511","Type":"ContainerStarted","Data":"d4616ebb68192b95bfea61bf9227d2a0d34c64630f5287106998dea9ac1320f4"}
Jul 02 23:21:08.002417 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:08.002241    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp" event={"ID":"144b0851-31ea-4c6d-b3cb-e7256b934511","Type":"ContainerStarted","Data":"307f7f37fe22267ad86afdb024995672e3c1a1acfb67cbb726e231aba95da2be"}
Jul 02 23:21:08.023217 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:08.023168    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp" podStartSLOduration=0.849722782 podStartE2EDuration="1.02315207s" totalImagesPullingTime="173.429288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:21:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:21:08.020451787 +0000 UTC m=+613.274241796" watchObservedRunningTime="2026-07-02 23:21:08.02315207 +0000 UTC m=+613.276942068"
Jul 02 23:21:13.402697 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:13.402644    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:13.403095 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:13.402735    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:13.415350 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:13.415327    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:14.044681 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:14.044650    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7ttdh"
Jul 02 23:21:15.777429 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:15.777392    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:21:17.601422 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:17.601333    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:17.601422 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:17.601386    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:17.613831 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:17.613804    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:21:18.064735 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:21:18.064700    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp"
Jul 02 23:22:43.485790 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:22:43.485751    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:22:53.284376 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:22:53.284294    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:23:05.073253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:23:05.073211    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:23:13.781687 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:23:13.781649    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:23:24.779234 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:23:24.779194    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:23:50.298941 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:23:50.298892    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:24:01.378497 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:24:01.378462    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:24:31.081895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:24:31.081795    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:24:40.382975 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:24:40.382932    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:24:48.693828 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:24:48.693787    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:24:59.275092 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:24:59.275048    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:25:08.888541 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:25:08.888503    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:25:17.679183 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:25:17.679147    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:25:29.689388 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:25:29.689350    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:25:38.779371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:25:38.779334    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:26:01.989101 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:26:01.989070    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:26:11.074952 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:26:11.074914    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:26:34.873164 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:26:34.873113    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:26:45.575217 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:26:45.575184    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:27:00.377922 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:27:00.377882    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:27:10.527763 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:27:10.527725    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:27:20.878952 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:27:20.877827    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:27:29.676389 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:27:29.676352    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:27:38.275912 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:27:38.275875    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:29:26.787583 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:29:26.787543    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:29:37.990101 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:29:37.990062    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:29:49.175432 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:29:49.175390    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:29:57.583735 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:29:57.583697    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:30:07.671539 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:30:07.671493    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:30:15.289409 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:30:15.289372    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:30:25.174247 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:30:25.174207    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:30:35.878367 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:30:35.878325    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:30:45.686447 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:30:45.686407    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:30:56.574877 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:30:56.574843    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:31:07.674109 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:31:07.674073    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:31:14.784946 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:31:14.784905    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:31:38.580869 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:31:38.580830    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:31:47.418366 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:31:47.418283    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:31:57.080478 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:31:57.080443    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:32:08.876098 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:32:08.876057    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:32:25.783824 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:32:25.783782    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:32:35.275114 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:32:35.275079    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:32:44.177493 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:32:44.177455    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:32:54.182834 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:32:54.182795    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:33:02.784498 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:33:02.784452    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:33:11.781241 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:33:11.781202    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:33:24.275900 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:33:24.275815    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:33:33.877624 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:33:33.877592    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:33:44.980678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:33:44.980641    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:33:54.284095 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:33:54.284056    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:34:06.472755 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:34:06.472723    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:34:13.982335 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:34:13.982300    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:34:25.677057 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:34:25.677015    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:34:37.184687 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:34:37.184648    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:34:50.575389 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:34:50.575351    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:35:03.794971 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:35:03.794934    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:35:14.077364 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:35:14.077320    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:35:24.384399 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:35:24.384365    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:35:36.583812 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:35:36.583775    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:35:47.811319 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:35:47.811228    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:35:57.480278 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:35:57.480246    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:36:06.994071 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:36:06.994032    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:37:25.578794 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:37:25.578749    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:37:34.777067 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:37:34.777032    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:37:46.291944 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:37:46.291906    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:37:57.099803 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:37:57.099761    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:38:23.479620 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:23.479581    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:38:38.590628 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.590590    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"]
Jul 02 23:38:38.594034 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.594017    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.596856 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.596829    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-bebb6de2\"/\"openshift-service-ca.crt\""
Jul 02 23:38:38.596992 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.596910    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-bebb6de2\"/\"kube-root-ca.crt\""
Jul 02 23:38:38.598157 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.598113    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-bebb6de2\"/\"default-dockercfg-szq9m\""
Jul 02 23:38:38.598237 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.598178    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-bebb6de2\"/\"e2e-lifecycle-model-bebb6de2-kserve-self-signed-certs\""
Jul 02 23:38:38.603192 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.603159    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"]
Jul 02 23:38:38.676487 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.676444    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb10b3d-75f1-4e0e-b461-e420c20120e8-tls-certs\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.676689 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.676512    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-tmp-dir\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.676689 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.676572    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-home\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.676689 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.676609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxbxz\" (UniqueName: \"kubernetes.io/projected/3fb10b3d-75f1-4e0e-b461-e420c20120e8-kube-api-access-lxbxz\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.676689 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.676682    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-dshm\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.676891 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.676747    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-model-cache\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.777978 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.777931    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxbxz\" (UniqueName: \"kubernetes.io/projected/3fb10b3d-75f1-4e0e-b461-e420c20120e8-kube-api-access-lxbxz\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.777978 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.777984    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-dshm\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.778281 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.778021    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-model-cache\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.778281 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.778044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb10b3d-75f1-4e0e-b461-e420c20120e8-tls-certs\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.778281 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.778100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-tmp-dir\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.778281 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.778203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-home\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.778495 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.778469    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-model-cache\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.778635 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.778614    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-tmp-dir\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.778699 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.778658    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-home\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.780430 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.780402    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-dshm\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.780553 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.780537    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb10b3d-75f1-4e0e-b461-e420c20120e8-tls-certs\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.788509 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.788476    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxbxz\" (UniqueName: \"kubernetes.io/projected/3fb10b3d-75f1-4e0e-b461-e420c20120e8-kube-api-access-lxbxz\") pod \"e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") " pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:38.904658 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:38.904547    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:39.041151 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:39.041108    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"]
Jul 02 23:38:39.043286 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:38:39.043253    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3fb10b3d_75f1_4e0e_b461_e420c20120e8.slice/crio-0cb82dead80fa318aef3b30ee5ee92d8fe9965f7b8e33cbf75279ed9899f7b8d WatchSource:0}: Error finding container 0cb82dead80fa318aef3b30ee5ee92d8fe9965f7b8e33cbf75279ed9899f7b8d: Status 404 returned error can't find the container with id 0cb82dead80fa318aef3b30ee5ee92d8fe9965f7b8e33cbf75279ed9899f7b8d
Jul 02 23:38:39.259157 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:39.259107    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7" event={"ID":"3fb10b3d-75f1-4e0e-b461-e420c20120e8","Type":"ContainerStarted","Data":"9533879c98195ca526232819cb42062c7db4a7323f3b9521949c352a78d9397e"}
Jul 02 23:38:39.259157 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:39.259162    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7" event={"ID":"3fb10b3d-75f1-4e0e-b461-e420c20120e8","Type":"ContainerStarted","Data":"0cb82dead80fa318aef3b30ee5ee92d8fe9965f7b8e33cbf75279ed9899f7b8d"}
Jul 02 23:38:39.276332 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:39.276277    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7" podStartSLOduration=1.276261122 podStartE2EDuration="1.276261122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:38:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:38:39.275107063 +0000 UTC m=+1664.528897087" watchObservedRunningTime="2026-07-02 23:38:39.276261122 +0000 UTC m=+1664.530051120"
Jul 02 23:38:41.781136 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:41.781091    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:38:48.905019 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:48.904974    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:48.905019 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:48.905042    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:48.917689 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:48.917663    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:49.318292 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:49.318257    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:38:55.078446 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:38:55.078401    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:39:08.678415 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:08.678377    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:39:09.024989 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:09.024954    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"]
Jul 02 23:39:24.045306 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.045220    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7" podUID="3fb10b3d-75f1-4e0e-b461-e420c20120e8" containerName="main" containerID="cri-o://9533879c98195ca526232819cb42062c7db4a7323f3b9521949c352a78d9397e" gracePeriod=45
Jul 02 23:39:24.304755 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.304691    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:39:24.377684 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.377649    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-home\") pod \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") "
Jul 02 23:39:24.377883 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.377716    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxbxz\" (UniqueName: \"kubernetes.io/projected/3fb10b3d-75f1-4e0e-b461-e420c20120e8-kube-api-access-lxbxz\") pod \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") "
Jul 02 23:39:24.377883 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.377744    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-dshm\") pod \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") "
Jul 02 23:39:24.377883 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.377774    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-model-cache\") pod \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") "
Jul 02 23:39:24.377883 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.377801    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-tmp-dir\") pod \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") "
Jul 02 23:39:24.377883 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.377868    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb10b3d-75f1-4e0e-b461-e420c20120e8-tls-certs\") pod \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\" (UID: \"3fb10b3d-75f1-4e0e-b461-e420c20120e8\") "
Jul 02 23:39:24.378186 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.377957    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-home" (OuterVolumeSpecName: "home") pod "3fb10b3d-75f1-4e0e-b461-e420c20120e8" (UID: "3fb10b3d-75f1-4e0e-b461-e420c20120e8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:39:24.378186 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.378072    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-model-cache" (OuterVolumeSpecName: "model-cache") pod "3fb10b3d-75f1-4e0e-b461-e420c20120e8" (UID: "3fb10b3d-75f1-4e0e-b461-e420c20120e8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:39:24.378186 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.378094    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3fb10b3d-75f1-4e0e-b461-e420c20120e8" (UID: "3fb10b3d-75f1-4e0e-b461-e420c20120e8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:39:24.378186 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.378181    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-home\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:39:24.378329 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.378195    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-model-cache\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:39:24.378329 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.378206    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-tmp-dir\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:39:24.379983 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.379962    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-dshm" (OuterVolumeSpecName: "dshm") pod "3fb10b3d-75f1-4e0e-b461-e420c20120e8" (UID: "3fb10b3d-75f1-4e0e-b461-e420c20120e8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:39:24.380068 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.380049    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fb10b3d-75f1-4e0e-b461-e420c20120e8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3fb10b3d-75f1-4e0e-b461-e420c20120e8" (UID: "3fb10b3d-75f1-4e0e-b461-e420c20120e8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 23:39:24.380109 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.380055    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fb10b3d-75f1-4e0e-b461-e420c20120e8-kube-api-access-lxbxz" (OuterVolumeSpecName: "kube-api-access-lxbxz") pod "3fb10b3d-75f1-4e0e-b461-e420c20120e8" (UID: "3fb10b3d-75f1-4e0e-b461-e420c20120e8"). InnerVolumeSpecName "kube-api-access-lxbxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:39:24.479365 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.479326    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3fb10b3d-75f1-4e0e-b461-e420c20120e8-tls-certs\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:39:24.479365 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.479362    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxbxz\" (UniqueName: \"kubernetes.io/projected/3fb10b3d-75f1-4e0e-b461-e420c20120e8-kube-api-access-lxbxz\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:39:24.479365 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.479371    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3fb10b3d-75f1-4e0e-b461-e420c20120e8-dshm\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:39:24.492475 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.492437    2580 generic.go:356] "Generic (PLEG): container finished" podID="3fb10b3d-75f1-4e0e-b461-e420c20120e8" containerID="9533879c98195ca526232819cb42062c7db4a7323f3b9521949c352a78d9397e" exitCode=0
Jul 02 23:39:24.492632 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.492482    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7" event={"ID":"3fb10b3d-75f1-4e0e-b461-e420c20120e8","Type":"ContainerDied","Data":"9533879c98195ca526232819cb42062c7db4a7323f3b9521949c352a78d9397e"}
Jul 02 23:39:24.492632 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.492497    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"
Jul 02 23:39:24.492632 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.492524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7" event={"ID":"3fb10b3d-75f1-4e0e-b461-e420c20120e8","Type":"ContainerDied","Data":"0cb82dead80fa318aef3b30ee5ee92d8fe9965f7b8e33cbf75279ed9899f7b8d"}
Jul 02 23:39:24.492632 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.492548    2580 scope.go:117] "RemoveContainer" containerID="9533879c98195ca526232819cb42062c7db4a7323f3b9521949c352a78d9397e"
Jul 02 23:39:24.501666 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.501646    2580 scope.go:117] "RemoveContainer" containerID="9533879c98195ca526232819cb42062c7db4a7323f3b9521949c352a78d9397e"
Jul 02 23:39:24.501944 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:39:24.501921    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9533879c98195ca526232819cb42062c7db4a7323f3b9521949c352a78d9397e\": container with ID starting with 9533879c98195ca526232819cb42062c7db4a7323f3b9521949c352a78d9397e not found: ID does not exist" containerID="9533879c98195ca526232819cb42062c7db4a7323f3b9521949c352a78d9397e"
Jul 02 23:39:24.502013 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.501952    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9533879c98195ca526232819cb42062c7db4a7323f3b9521949c352a78d9397e"} err="failed to get container status \"9533879c98195ca526232819cb42062c7db4a7323f3b9521949c352a78d9397e\": rpc error: code = NotFound desc = could not find container \"9533879c98195ca526232819cb42062c7db4a7323f3b9521949c352a78d9397e\": container with ID starting with 9533879c98195ca526232819cb42062c7db4a7323f3b9521949c352a78d9397e not found: ID does not exist"
Jul 02 23:39:24.515052 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.515023    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"]
Jul 02 23:39:24.518002 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:24.517978    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-bebb6de2/e2e-lifecycle-model-bebb6de2-kserve-56646cb8c5-9gzg7"]
Jul 02 23:39:25.283465 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:25.283434    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fb10b3d-75f1-4e0e-b461-e420c20120e8" path="/var/lib/kubelet/pods/3fb10b3d-75f1-4e0e-b461-e420c20120e8/volumes"
Jul 02 23:39:48.777081 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:39:48.777047    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:40:01.489705 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:40:01.489658    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:40:10.576197 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:40:10.576152    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:40:54.578839 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:40:54.578801    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:41:07.093064 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.093027    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"]
Jul 02 23:41:07.093463 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.093416    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fb10b3d-75f1-4e0e-b461-e420c20120e8" containerName="main"
Jul 02 23:41:07.093463 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.093427    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fb10b3d-75f1-4e0e-b461-e420c20120e8" containerName="main"
Jul 02 23:41:07.093548 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.093500    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fb10b3d-75f1-4e0e-b461-e420c20120e8" containerName="main"
Jul 02 23:41:07.096442 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.096422    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.100561 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.100540    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d73038\"/\"test-model-d73038-kserve-self-signed-certs\""
Jul 02 23:41:07.100675 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.100589    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d73038\"/\"kube-root-ca.crt\""
Jul 02 23:41:07.100675 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.100594    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-d73038\"/\"default-dockercfg-8kzl7\""
Jul 02 23:41:07.100675 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.100594    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-d73038\"/\"openshift-service-ca.crt\""
Jul 02 23:41:07.106834 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.106812    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"]
Jul 02 23:41:07.174764 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.174723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-tls-certs\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.174764 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.174770    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-dshm\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.175008 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.174790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-home\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.175008 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.174931    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-tmp-dir\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.175008 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.174965    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-model-cache\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.175008 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.174984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs9hk\" (UniqueName: \"kubernetes.io/projected/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-kube-api-access-vs9hk\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.275718 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.275687    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-home\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.275887 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.275747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-tmp-dir\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.275887 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.275769    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-model-cache\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.275887 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.275786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs9hk\" (UniqueName: \"kubernetes.io/projected/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-kube-api-access-vs9hk\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.275887 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.275859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-tls-certs\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.275887 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.275888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-dshm\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.276192 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.276160    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-home\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.276192 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.276181    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-tmp-dir\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.276374 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.276281    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-model-cache\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.278226 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.278204    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-dshm\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.278532 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.278511    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-tls-certs\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.284903 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.284877    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs9hk\" (UniqueName: \"kubernetes.io/projected/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-kube-api-access-vs9hk\") pod \"test-model-d73038-kserve-59bf6f4cf-6fh8p\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") " pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.406111 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.406023    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:07.568142 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.568099    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"]
Jul 02 23:41:07.570611 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:41:07.570584    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc5ebb54_e8d3_41c7_a967_c73cf0d60526.slice/crio-d3bc4f13796f58e7f78d6dfddf8a2b398189185cf2c1be39e5d01971975582a1 WatchSource:0}: Error finding container d3bc4f13796f58e7f78d6dfddf8a2b398189185cf2c1be39e5d01971975582a1: Status 404 returned error can't find the container with id d3bc4f13796f58e7f78d6dfddf8a2b398189185cf2c1be39e5d01971975582a1
Jul 02 23:41:07.994765 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.994728    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p" event={"ID":"cc5ebb54-e8d3-41c7-a967-c73cf0d60526","Type":"ContainerStarted","Data":"d1fa8bcf0e0b1a10595886d006d6b3fb6678d383fdafff227aff48996da2d41f"}
Jul 02 23:41:07.994765 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:07.994774    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p" event={"ID":"cc5ebb54-e8d3-41c7-a967-c73cf0d60526","Type":"ContainerStarted","Data":"d3bc4f13796f58e7f78d6dfddf8a2b398189185cf2c1be39e5d01971975582a1"}
Jul 02 23:41:08.014590 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:08.014540    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p" podStartSLOduration=1.014525584 podStartE2EDuration="1.014525584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:41:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:41:08.012802749 +0000 UTC m=+1813.266592750" watchObservedRunningTime="2026-07-02 23:41:08.014525584 +0000 UTC m=+1813.268315583"
Jul 02 23:41:09.987772 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:09.987732    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:41:17.406720 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:17.406676    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:17.406720 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:17.406726    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:17.419032 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:17.419001    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:18.053574 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:18.053542    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:41:24.680525 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:24.680485    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:41:29.990183 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:29.990150    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"]
Jul 02 23:41:29.993306 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:29.993286    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:29.997892 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:29.997872    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8c4f9c\"/\"test-model-8c4f9c-kserve-self-signed-certs\""
Jul 02 23:41:29.997981 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:29.997876    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8c4f9c\"/\"default-dockercfg-c765c\""
Jul 02 23:41:29.998067 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:29.998052    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8c4f9c\"/\"openshift-service-ca.crt\""
Jul 02 23:41:29.998284 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:29.998265    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8c4f9c\"/\"kube-root-ca.crt\""
Jul 02 23:41:30.006798 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.006768    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"]
Jul 02 23:41:30.080995 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.080955    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d439a273-c77f-42ff-af2d-0f7704752fa7-tls-certs\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.080995 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.081001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nknm\" (UniqueName: \"kubernetes.io/projected/d439a273-c77f-42ff-af2d-0f7704752fa7-kube-api-access-7nknm\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.081235 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.081025    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-home\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.081235 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.081055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-tmp-dir\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.081235 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.081098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-dshm\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.081235 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.081158    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-model-cache\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.181765 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.181731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-model-cache\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.181973 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.181789    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d439a273-c77f-42ff-af2d-0f7704752fa7-tls-certs\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.181973 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.181822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nknm\" (UniqueName: \"kubernetes.io/projected/d439a273-c77f-42ff-af2d-0f7704752fa7-kube-api-access-7nknm\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.181973 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.181858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-home\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.181973 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.181880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-tmp-dir\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.181973 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.181927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-dshm\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.182275 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.182250    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-model-cache\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.182334 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.182258    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-tmp-dir\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.182384 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.182323    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-home\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.184144 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.184097    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-dshm\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.184500 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.184481    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d439a273-c77f-42ff-af2d-0f7704752fa7-tls-certs\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.192998 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.192974    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nknm\" (UniqueName: \"kubernetes.io/projected/d439a273-c77f-42ff-af2d-0f7704752fa7-kube-api-access-7nknm\") pod \"test-model-8c4f9c-kserve-f8cccc675-gh2wf\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") " pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.306830 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.306731    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:30.439768 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:30.439730    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"]
Jul 02 23:41:30.442106 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:41:30.442077    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd439a273_c77f_42ff_af2d_0f7704752fa7.slice/crio-502c5ddc74e2525b4af93be49e5e218ec6cf02ea147628eda04c0f6d76f43005 WatchSource:0}: Error finding container 502c5ddc74e2525b4af93be49e5e218ec6cf02ea147628eda04c0f6d76f43005: Status 404 returned error can't find the container with id 502c5ddc74e2525b4af93be49e5e218ec6cf02ea147628eda04c0f6d76f43005
Jul 02 23:41:31.110230 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:31.110191    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf" event={"ID":"d439a273-c77f-42ff-af2d-0f7704752fa7","Type":"ContainerStarted","Data":"d3566a9a70397cdc52a66c7394e79b9369d74fb134ca290216f75d8152c79a17"}
Jul 02 23:41:31.110230 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:31.110233    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf" event={"ID":"d439a273-c77f-42ff-af2d-0f7704752fa7","Type":"ContainerStarted","Data":"502c5ddc74e2525b4af93be49e5e218ec6cf02ea147628eda04c0f6d76f43005"}
Jul 02 23:41:31.126271 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:31.126226    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf" podStartSLOduration=2.126210263 podStartE2EDuration="2.126210263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:41:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:41:31.125535736 +0000 UTC m=+1836.379325732" watchObservedRunningTime="2026-07-02 23:41:31.126210263 +0000 UTC m=+1836.380000263"
Jul 02 23:41:33.980481 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:33.980447    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:41:40.307252 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:40.307196    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:40.307252 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:40.307264    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:40.319901 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:40.319875    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:41.170496 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:41.170467    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:41:47.981511 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:47.981476    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:41:49.370931 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:49.370900    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"]
Jul 02 23:41:50.269601 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:41:50.269570    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"]
Jul 02 23:42:01.771845 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:01.771808    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:42:04.390991 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.390945    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p" podUID="cc5ebb54-e8d3-41c7-a967-c73cf0d60526" containerName="main" containerID="cri-o://d1fa8bcf0e0b1a10595886d006d6b3fb6678d383fdafff227aff48996da2d41f" gracePeriod=45
Jul 02 23:42:04.642291 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.642229    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:42:04.672763 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.672736    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-home\") pod \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") "
Jul 02 23:42:04.672910 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.672780    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-tls-certs\") pod \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") "
Jul 02 23:42:04.672910 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.672799    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-model-cache\") pod \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") "
Jul 02 23:42:04.672910 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.672833    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vs9hk\" (UniqueName: \"kubernetes.io/projected/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-kube-api-access-vs9hk\") pod \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") "
Jul 02 23:42:04.672910 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.672871    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-dshm\") pod \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") "
Jul 02 23:42:04.672910 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.672900    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-tmp-dir\") pod \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\" (UID: \"cc5ebb54-e8d3-41c7-a967-c73cf0d60526\") "
Jul 02 23:42:04.673190 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.672994    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-home" (OuterVolumeSpecName: "home") pod "cc5ebb54-e8d3-41c7-a967-c73cf0d60526" (UID: "cc5ebb54-e8d3-41c7-a967-c73cf0d60526"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:42:04.673190 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.673139    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-model-cache" (OuterVolumeSpecName: "model-cache") pod "cc5ebb54-e8d3-41c7-a967-c73cf0d60526" (UID: "cc5ebb54-e8d3-41c7-a967-c73cf0d60526"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:42:04.673301 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.673259    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-home\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:04.673301 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.673277    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-model-cache\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:04.673408 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.673310    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cc5ebb54-e8d3-41c7-a967-c73cf0d60526" (UID: "cc5ebb54-e8d3-41c7-a967-c73cf0d60526"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:42:04.675218 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.675190    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-dshm" (OuterVolumeSpecName: "dshm") pod "cc5ebb54-e8d3-41c7-a967-c73cf0d60526" (UID: "cc5ebb54-e8d3-41c7-a967-c73cf0d60526"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:42:04.675323 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.675192    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-kube-api-access-vs9hk" (OuterVolumeSpecName: "kube-api-access-vs9hk") pod "cc5ebb54-e8d3-41c7-a967-c73cf0d60526" (UID: "cc5ebb54-e8d3-41c7-a967-c73cf0d60526"). InnerVolumeSpecName "kube-api-access-vs9hk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:42:04.675323 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.675198    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cc5ebb54-e8d3-41c7-a967-c73cf0d60526" (UID: "cc5ebb54-e8d3-41c7-a967-c73cf0d60526"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 23:42:04.773903 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.773869    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-dshm\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:04.773903 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.773899    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-tmp-dir\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:04.773903 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.773910    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-tls-certs\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:04.774166 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:04.773925    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vs9hk\" (UniqueName: \"kubernetes.io/projected/cc5ebb54-e8d3-41c7-a967-c73cf0d60526-kube-api-access-vs9hk\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:05.277150 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.277096    2580 generic.go:356] "Generic (PLEG): container finished" podID="cc5ebb54-e8d3-41c7-a967-c73cf0d60526" containerID="d1fa8bcf0e0b1a10595886d006d6b3fb6678d383fdafff227aff48996da2d41f" exitCode=0
Jul 02 23:42:05.279405 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.279383    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"
Jul 02 23:42:05.289993 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.289963    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p" event={"ID":"cc5ebb54-e8d3-41c7-a967-c73cf0d60526","Type":"ContainerDied","Data":"d1fa8bcf0e0b1a10595886d006d6b3fb6678d383fdafff227aff48996da2d41f"}
Jul 02 23:42:05.290155 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.290007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p" event={"ID":"cc5ebb54-e8d3-41c7-a967-c73cf0d60526","Type":"ContainerDied","Data":"d3bc4f13796f58e7f78d6dfddf8a2b398189185cf2c1be39e5d01971975582a1"}
Jul 02 23:42:05.290155 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.290031    2580 scope.go:117] "RemoveContainer" containerID="d1fa8bcf0e0b1a10595886d006d6b3fb6678d383fdafff227aff48996da2d41f"
Jul 02 23:42:05.290380 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.290352    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf" podUID="d439a273-c77f-42ff-af2d-0f7704752fa7" containerName="main" containerID="cri-o://d3566a9a70397cdc52a66c7394e79b9369d74fb134ca290216f75d8152c79a17" gracePeriod=45
Jul 02 23:42:05.299542 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.299527    2580 scope.go:117] "RemoveContainer" containerID="d1fa8bcf0e0b1a10595886d006d6b3fb6678d383fdafff227aff48996da2d41f"
Jul 02 23:42:05.299822 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:42:05.299792    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1fa8bcf0e0b1a10595886d006d6b3fb6678d383fdafff227aff48996da2d41f\": container with ID starting with d1fa8bcf0e0b1a10595886d006d6b3fb6678d383fdafff227aff48996da2d41f not found: ID does not exist" containerID="d1fa8bcf0e0b1a10595886d006d6b3fb6678d383fdafff227aff48996da2d41f"
Jul 02 23:42:05.299926 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.299832    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1fa8bcf0e0b1a10595886d006d6b3fb6678d383fdafff227aff48996da2d41f"} err="failed to get container status \"d1fa8bcf0e0b1a10595886d006d6b3fb6678d383fdafff227aff48996da2d41f\": rpc error: code = NotFound desc = could not find container \"d1fa8bcf0e0b1a10595886d006d6b3fb6678d383fdafff227aff48996da2d41f\": container with ID starting with d1fa8bcf0e0b1a10595886d006d6b3fb6678d383fdafff227aff48996da2d41f not found: ID does not exist"
Jul 02 23:42:05.308389 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.308366    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"]
Jul 02 23:42:05.312335 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.312315    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-d73038/test-model-d73038-kserve-59bf6f4cf-6fh8p"]
Jul 02 23:42:05.556076 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.556049    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:42:05.579320 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.579291    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-home\") pod \"d439a273-c77f-42ff-af2d-0f7704752fa7\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") "
Jul 02 23:42:05.579473 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.579332    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d439a273-c77f-42ff-af2d-0f7704752fa7-tls-certs\") pod \"d439a273-c77f-42ff-af2d-0f7704752fa7\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") "
Jul 02 23:42:05.579473 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.579362    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nknm\" (UniqueName: \"kubernetes.io/projected/d439a273-c77f-42ff-af2d-0f7704752fa7-kube-api-access-7nknm\") pod \"d439a273-c77f-42ff-af2d-0f7704752fa7\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") "
Jul 02 23:42:05.579473 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.579392    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-dshm\") pod \"d439a273-c77f-42ff-af2d-0f7704752fa7\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") "
Jul 02 23:42:05.579650 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.579506    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-model-cache\") pod \"d439a273-c77f-42ff-af2d-0f7704752fa7\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") "
Jul 02 23:42:05.579650 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.579556    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-tmp-dir\") pod \"d439a273-c77f-42ff-af2d-0f7704752fa7\" (UID: \"d439a273-c77f-42ff-af2d-0f7704752fa7\") "
Jul 02 23:42:05.579650 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.579590    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-home" (OuterVolumeSpecName: "home") pod "d439a273-c77f-42ff-af2d-0f7704752fa7" (UID: "d439a273-c77f-42ff-af2d-0f7704752fa7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:42:05.579820 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.579778    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-model-cache" (OuterVolumeSpecName: "model-cache") pod "d439a273-c77f-42ff-af2d-0f7704752fa7" (UID: "d439a273-c77f-42ff-af2d-0f7704752fa7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:42:05.579889 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.579854    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-model-cache\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:05.579945 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.579896    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-home\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:05.579945 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.579921    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d439a273-c77f-42ff-af2d-0f7704752fa7" (UID: "d439a273-c77f-42ff-af2d-0f7704752fa7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:42:05.581617 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.581594    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d439a273-c77f-42ff-af2d-0f7704752fa7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d439a273-c77f-42ff-af2d-0f7704752fa7" (UID: "d439a273-c77f-42ff-af2d-0f7704752fa7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 23:42:05.581719 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.581617    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-dshm" (OuterVolumeSpecName: "dshm") pod "d439a273-c77f-42ff-af2d-0f7704752fa7" (UID: "d439a273-c77f-42ff-af2d-0f7704752fa7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:42:05.581825 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.581807    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d439a273-c77f-42ff-af2d-0f7704752fa7-kube-api-access-7nknm" (OuterVolumeSpecName: "kube-api-access-7nknm") pod "d439a273-c77f-42ff-af2d-0f7704752fa7" (UID: "d439a273-c77f-42ff-af2d-0f7704752fa7"). InnerVolumeSpecName "kube-api-access-7nknm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:42:05.680878 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.680849    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-tmp-dir\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:05.680878 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.680873    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d439a273-c77f-42ff-af2d-0f7704752fa7-tls-certs\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:05.680878 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.680884    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nknm\" (UniqueName: \"kubernetes.io/projected/d439a273-c77f-42ff-af2d-0f7704752fa7-kube-api-access-7nknm\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:05.681084 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:05.680893    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d439a273-c77f-42ff-af2d-0f7704752fa7-dshm\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:06.283419 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:06.283387    2580 generic.go:356] "Generic (PLEG): container finished" podID="d439a273-c77f-42ff-af2d-0f7704752fa7" containerID="d3566a9a70397cdc52a66c7394e79b9369d74fb134ca290216f75d8152c79a17" exitCode=0
Jul 02 23:42:06.283633 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:06.283432    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"
Jul 02 23:42:06.283633 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:06.283473    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf" event={"ID":"d439a273-c77f-42ff-af2d-0f7704752fa7","Type":"ContainerDied","Data":"d3566a9a70397cdc52a66c7394e79b9369d74fb134ca290216f75d8152c79a17"}
Jul 02 23:42:06.283633 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:06.283521    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf" event={"ID":"d439a273-c77f-42ff-af2d-0f7704752fa7","Type":"ContainerDied","Data":"502c5ddc74e2525b4af93be49e5e218ec6cf02ea147628eda04c0f6d76f43005"}
Jul 02 23:42:06.283633 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:06.283545    2580 scope.go:117] "RemoveContainer" containerID="d3566a9a70397cdc52a66c7394e79b9369d74fb134ca290216f75d8152c79a17"
Jul 02 23:42:06.292690 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:06.292675    2580 scope.go:117] "RemoveContainer" containerID="d3566a9a70397cdc52a66c7394e79b9369d74fb134ca290216f75d8152c79a17"
Jul 02 23:42:06.292930 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:42:06.292910    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3566a9a70397cdc52a66c7394e79b9369d74fb134ca290216f75d8152c79a17\": container with ID starting with d3566a9a70397cdc52a66c7394e79b9369d74fb134ca290216f75d8152c79a17 not found: ID does not exist" containerID="d3566a9a70397cdc52a66c7394e79b9369d74fb134ca290216f75d8152c79a17"
Jul 02 23:42:06.293004 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:06.292963    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3566a9a70397cdc52a66c7394e79b9369d74fb134ca290216f75d8152c79a17"} err="failed to get container status \"d3566a9a70397cdc52a66c7394e79b9369d74fb134ca290216f75d8152c79a17\": rpc error: code = NotFound desc = could not find container \"d3566a9a70397cdc52a66c7394e79b9369d74fb134ca290216f75d8152c79a17\": container with ID starting with d3566a9a70397cdc52a66c7394e79b9369d74fb134ca290216f75d8152c79a17 not found: ID does not exist"
Jul 02 23:42:06.304323 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:06.304296    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"]
Jul 02 23:42:06.307831 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:06.307809    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-8c4f9c/test-model-8c4f9c-kserve-f8cccc675-gh2wf"]
Jul 02 23:42:07.284217 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:07.284187    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc5ebb54-e8d3-41c7-a967-c73cf0d60526" path="/var/lib/kubelet/pods/cc5ebb54-e8d3-41c7-a967-c73cf0d60526/volumes"
Jul 02 23:42:07.284592 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:07.284579    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d439a273-c77f-42ff-af2d-0f7704752fa7" path="/var/lib/kubelet/pods/d439a273-c77f-42ff-af2d-0f7704752fa7/volumes"
Jul 02 23:42:12.784692 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:12.784656    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:42:15.936553 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.936514    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"]
Jul 02 23:42:15.937054 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.937032    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d439a273-c77f-42ff-af2d-0f7704752fa7" containerName="main"
Jul 02 23:42:15.937142 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.937057    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d439a273-c77f-42ff-af2d-0f7704752fa7" containerName="main"
Jul 02 23:42:15.937142 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.937086    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc5ebb54-e8d3-41c7-a967-c73cf0d60526" containerName="main"
Jul 02 23:42:15.937142 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.937095    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc5ebb54-e8d3-41c7-a967-c73cf0d60526" containerName="main"
Jul 02 23:42:15.937304 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.937202    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d439a273-c77f-42ff-af2d-0f7704752fa7" containerName="main"
Jul 02 23:42:15.937304 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.937218    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc5ebb54-e8d3-41c7-a967-c73cf0d60526" containerName="main"
Jul 02 23:42:15.941590 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.941566    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:15.944437 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.944406    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-79bbe8-openshift-default-dockercfg-vmvs2\""
Jul 02 23:42:15.953429 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.953409    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"]
Jul 02 23:42:15.968849 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.968823    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-workload-socket\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:15.968980 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.968863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54158b19-17a7-4756-a060-61966d83d9ce-istiod-ca-cert\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:15.968980 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.968887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-istio-envoy\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:15.968980 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.968931    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54158b19-17a7-4756-a060-61966d83d9ce-istio-token\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:15.969172 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.969012    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54158b19-17a7-4756-a060-61966d83d9ce-istio-podinfo\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:15.969172 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.969032    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48z6q\" (UniqueName: \"kubernetes.io/projected/54158b19-17a7-4756-a060-61966d83d9ce-kube-api-access-48z6q\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:15.969172 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.969062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-istio-data\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:15.969172 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.969098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-credential-socket\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:15.969172 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:15.969113    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-workload-certs\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.070475 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.070438    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-istio-data\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.070675 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.070493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-credential-socket\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.070675 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.070520    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-workload-certs\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.070675 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.070550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-workload-socket\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.070837 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.070699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54158b19-17a7-4756-a060-61966d83d9ce-istiod-ca-cert\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.070837 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.070742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-istio-envoy\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.070837 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.070770    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54158b19-17a7-4756-a060-61966d83d9ce-istio-token\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.071001 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.070867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54158b19-17a7-4756-a060-61966d83d9ce-istio-podinfo\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.071001 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.070895    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48z6q\" (UniqueName: \"kubernetes.io/projected/54158b19-17a7-4756-a060-61966d83d9ce-kube-api-access-48z6q\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.071132 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.071029    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-workload-socket\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.071132 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.071053    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-credential-socket\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.071245 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.071203    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-workload-certs\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.071385 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.071361    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-istio-data\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.071658 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.071636    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54158b19-17a7-4756-a060-61966d83d9ce-istiod-ca-cert\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.073110 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.073092    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-istio-envoy\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.073480 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.073460    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54158b19-17a7-4756-a060-61966d83d9ce-istio-podinfo\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.079371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.079350    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54158b19-17a7-4756-a060-61966d83d9ce-istio-token\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.079476 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.079458    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48z6q\" (UniqueName: \"kubernetes.io/projected/54158b19-17a7-4756-a060-61966d83d9ce-kube-api-access-48z6q\") pod \"e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") " pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.252163 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.252046    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:16.399341 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.399317    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"]
Jul 02 23:42:16.400998 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:42:16.400970    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54158b19_17a7_4756_a060_61966d83d9ce.slice/crio-bd4bdaa6d555a8e5da057576d95e68bf020ef5f41303fa193bbe3fda8962fd7a WatchSource:0}: Error finding container bd4bdaa6d555a8e5da057576d95e68bf020ef5f41303fa193bbe3fda8962fd7a: Status 404 returned error can't find the container with id bd4bdaa6d555a8e5da057576d95e68bf020ef5f41303fa193bbe3fda8962fd7a
Jul 02 23:42:16.403160 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.403103    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:42:16.403280 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.403203    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:42:16.403280 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:16.403245    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:42:17.343648 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:17.343548    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf" event={"ID":"54158b19-17a7-4756-a060-61966d83d9ce","Type":"ContainerStarted","Data":"5683114e60a8783464f427272ab3ce84a011a145cc38cb9f610b0cdebff43c1e"}
Jul 02 23:42:17.343648 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:17.343620    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf" event={"ID":"54158b19-17a7-4756-a060-61966d83d9ce","Type":"ContainerStarted","Data":"bd4bdaa6d555a8e5da057576d95e68bf020ef5f41303fa193bbe3fda8962fd7a"}
Jul 02 23:42:17.369015 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:17.368958    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf" podStartSLOduration=2.368941866 podStartE2EDuration="2.368941866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:42:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:42:17.367737539 +0000 UTC m=+1882.621527538" watchObservedRunningTime="2026-07-02 23:42:17.368941866 +0000 UTC m=+1882.622731865"
Jul 02 23:42:18.252765 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:18.252726    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:18.257701 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:18.257675    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:18.348696 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:18.348652    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:18.349574 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:18.349554    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:23.783371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:23.783338    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:42:32.421481 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:32.421445    2580 generic.go:356] "Generic (PLEG): container finished" podID="54158b19-17a7-4756-a060-61966d83d9ce" containerID="5683114e60a8783464f427272ab3ce84a011a145cc38cb9f610b0cdebff43c1e" exitCode=0
Jul 02 23:42:32.422012 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:32.421513    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf" event={"ID":"54158b19-17a7-4756-a060-61966d83d9ce","Type":"ContainerDied","Data":"5683114e60a8783464f427272ab3ce84a011a145cc38cb9f610b0cdebff43c1e"}
Jul 02 23:42:32.422091 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:32.422010    2580 scope.go:117] "RemoveContainer" containerID="5683114e60a8783464f427272ab3ce84a011a145cc38cb9f610b0cdebff43c1e"
Jul 02 23:42:32.423090 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:32.423059    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:42:32.423247 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:32.423173    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:42:32.423247 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:32.423222    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:42:33.348749 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:33.348693    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:33.433609 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:33.433562    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf" event={"ID":"54158b19-17a7-4756-a060-61966d83d9ce","Type":"ContainerStarted","Data":"6fe5dc95a8ed97f2d81f8903dd47015ed8e72ef9a36c2bb7b0023055098c46f1"}
Jul 02 23:42:34.253229 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:34.253175    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:34.254554 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:34.254523    2580 patch_prober.go:28] interesting pod/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 02 23:42:34.254683 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:34.254578    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf" podUID="54158b19-17a7-4756-a060-61966d83d9ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 02 23:42:34.493763 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:34.493724    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:42:35.253500 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:35.253457    2580 patch_prober.go:28] interesting pod/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 02 23:42:35.253718 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:35.253521    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf" podUID="54158b19-17a7-4756-a060-61966d83d9ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 02 23:42:36.252790 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:36.252744    2580 patch_prober.go:28] interesting pod/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 02 23:42:36.253330 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:36.252837    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf" podUID="54158b19-17a7-4756-a060-61966d83d9ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 02 23:42:37.253443 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:37.253411    2580 patch_prober.go:28] interesting pod/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Jul 02 23:42:37.253822 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:37.253480    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf" podUID="54158b19-17a7-4756-a060-61966d83d9ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Jul 02 23:42:38.257578 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:38.257544    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:38.258006 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:38.257938    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:38.258645 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:38.258616    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:46.173891 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:46.173842    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"]
Jul 02 23:42:46.174408 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:46.174189    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf" podUID="54158b19-17a7-4756-a060-61966d83d9ce" containerName="istio-proxy" containerID="cri-o://6fe5dc95a8ed97f2d81f8903dd47015ed8e72ef9a36c2bb7b0023055098c46f1" gracePeriod=30
Jul 02 23:42:49.072412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:49.072376    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:42:51.536212 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.536098    2580 generic.go:356] "Generic (PLEG): container finished" podID="54158b19-17a7-4756-a060-61966d83d9ce" containerID="6fe5dc95a8ed97f2d81f8903dd47015ed8e72ef9a36c2bb7b0023055098c46f1" exitCode=0
Jul 02 23:42:51.536212 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.536176    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf" event={"ID":"54158b19-17a7-4756-a060-61966d83d9ce","Type":"ContainerDied","Data":"6fe5dc95a8ed97f2d81f8903dd47015ed8e72ef9a36c2bb7b0023055098c46f1"}
Jul 02 23:42:51.536681 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.536252    2580 scope.go:117] "RemoveContainer" containerID="5683114e60a8783464f427272ab3ce84a011a145cc38cb9f610b0cdebff43c1e"
Jul 02 23:42:51.840195 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.840165    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:51.991994 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.991960    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54158b19-17a7-4756-a060-61966d83d9ce-istio-podinfo\") pod \"54158b19-17a7-4756-a060-61966d83d9ce\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") "
Jul 02 23:42:51.992200 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.992008    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-istio-data\") pod \"54158b19-17a7-4756-a060-61966d83d9ce\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") "
Jul 02 23:42:51.992200 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.992030    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54158b19-17a7-4756-a060-61966d83d9ce-istio-token\") pod \"54158b19-17a7-4756-a060-61966d83d9ce\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") "
Jul 02 23:42:51.992200 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.992059    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-istio-envoy\") pod \"54158b19-17a7-4756-a060-61966d83d9ce\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") "
Jul 02 23:42:51.992378 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.992204    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54158b19-17a7-4756-a060-61966d83d9ce-istiod-ca-cert\") pod \"54158b19-17a7-4756-a060-61966d83d9ce\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") "
Jul 02 23:42:51.992378 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.992295    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48z6q\" (UniqueName: \"kubernetes.io/projected/54158b19-17a7-4756-a060-61966d83d9ce-kube-api-access-48z6q\") pod \"54158b19-17a7-4756-a060-61966d83d9ce\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") "
Jul 02 23:42:51.992378 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.992329    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-workload-socket\") pod \"54158b19-17a7-4756-a060-61966d83d9ce\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") "
Jul 02 23:42:51.992527 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.992391    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-workload-certs\") pod \"54158b19-17a7-4756-a060-61966d83d9ce\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") "
Jul 02 23:42:51.992527 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.992445    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-credential-socket\") pod \"54158b19-17a7-4756-a060-61966d83d9ce\" (UID: \"54158b19-17a7-4756-a060-61966d83d9ce\") "
Jul 02 23:42:51.992624 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.992604    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54158b19-17a7-4756-a060-61966d83d9ce-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "54158b19-17a7-4756-a060-61966d83d9ce" (UID: "54158b19-17a7-4756-a060-61966d83d9ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 23:42:51.992676 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.992657    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "54158b19-17a7-4756-a060-61966d83d9ce" (UID: "54158b19-17a7-4756-a060-61966d83d9ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:42:51.992845 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.992826    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54158b19-17a7-4756-a060-61966d83d9ce-istiod-ca-cert\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:51.992919 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.992850    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-workload-socket\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:51.992919 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.992829    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "54158b19-17a7-4756-a060-61966d83d9ce" (UID: "54158b19-17a7-4756-a060-61966d83d9ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:42:51.992919 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.992855    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "54158b19-17a7-4756-a060-61966d83d9ce" (UID: "54158b19-17a7-4756-a060-61966d83d9ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:42:51.993080 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.993058    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-istio-data" (OuterVolumeSpecName: "istio-data") pod "54158b19-17a7-4756-a060-61966d83d9ce" (UID: "54158b19-17a7-4756-a060-61966d83d9ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:42:51.994331 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.994303    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54158b19-17a7-4756-a060-61966d83d9ce-istio-token" (OuterVolumeSpecName: "istio-token") pod "54158b19-17a7-4756-a060-61966d83d9ce" (UID: "54158b19-17a7-4756-a060-61966d83d9ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:42:51.994517 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.994492    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/54158b19-17a7-4756-a060-61966d83d9ce-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "54158b19-17a7-4756-a060-61966d83d9ce" (UID: "54158b19-17a7-4756-a060-61966d83d9ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 23:42:51.994767 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.994751    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54158b19-17a7-4756-a060-61966d83d9ce-kube-api-access-48z6q" (OuterVolumeSpecName: "kube-api-access-48z6q") pod "54158b19-17a7-4756-a060-61966d83d9ce" (UID: "54158b19-17a7-4756-a060-61966d83d9ce"). InnerVolumeSpecName "kube-api-access-48z6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:42:51.994767 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:51.994755    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "54158b19-17a7-4756-a060-61966d83d9ce" (UID: "54158b19-17a7-4756-a060-61966d83d9ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:42:52.094219 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:52.094106    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-istio-data\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:52.094219 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:52.094154    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54158b19-17a7-4756-a060-61966d83d9ce-istio-token\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:52.094219 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:52.094163    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-istio-envoy\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:52.094219 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:52.094174    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48z6q\" (UniqueName: \"kubernetes.io/projected/54158b19-17a7-4756-a060-61966d83d9ce-kube-api-access-48z6q\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:52.094219 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:52.094183    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-workload-certs\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:52.094219 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:52.094191    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54158b19-17a7-4756-a060-61966d83d9ce-credential-socket\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:52.094219 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:52.094198    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54158b19-17a7-4756-a060-61966d83d9ce-istio-podinfo\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:42:52.541902 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:52.541869    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"
Jul 02 23:42:52.541902 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:52.541886    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf" event={"ID":"54158b19-17a7-4756-a060-61966d83d9ce","Type":"ContainerDied","Data":"bd4bdaa6d555a8e5da057576d95e68bf020ef5f41303fa193bbe3fda8962fd7a"}
Jul 02 23:42:52.542401 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:52.541942    2580 scope.go:117] "RemoveContainer" containerID="6fe5dc95a8ed97f2d81f8903dd47015ed8e72ef9a36c2bb7b0023055098c46f1"
Jul 02 23:42:52.563482 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:52.563453    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"]
Jul 02 23:42:52.567102 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:52.567080    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-79bbe8-openshift-default-75f4f74765-h59pf"]
Jul 02 23:42:53.283808 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:42:53.283775    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54158b19-17a7-4756-a060-61966d83d9ce" path="/var/lib/kubelet/pods/54158b19-17a7-4756-a060-61966d83d9ce/volumes"
Jul 02 23:43:01.181301 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:01.181262    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:43:02.418815 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.418784    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"]
Jul 02 23:43:02.419282 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.419171    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54158b19-17a7-4756-a060-61966d83d9ce" containerName="istio-proxy"
Jul 02 23:43:02.419282 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.419185    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="54158b19-17a7-4756-a060-61966d83d9ce" containerName="istio-proxy"
Jul 02 23:43:02.419282 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.419254    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="54158b19-17a7-4756-a060-61966d83d9ce" containerName="istio-proxy"
Jul 02 23:43:02.419282 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.419263    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="54158b19-17a7-4756-a060-61966d83d9ce" containerName="istio-proxy"
Jul 02 23:43:02.422410 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.422385    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.425050 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.425026    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-fcd282-openshift-default-dockercfg-qgnwv\""
Jul 02 23:43:02.436292 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.436266    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"]
Jul 02 23:43:02.484665 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.484632    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-data\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.484836 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.484682    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-credential-socket\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.484836 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.484699    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-token\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.484836 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.484726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-workload-socket\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.484836 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.484750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-podinfo\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.484836 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.484769    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acd5b194-6f4c-4cef-9342-95ec643504a8-istiod-ca-cert\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.484836 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.484805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-workload-certs\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.485041 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.484859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-envoy\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.485041 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.484876    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm5dp\" (UniqueName: \"kubernetes.io/projected/acd5b194-6f4c-4cef-9342-95ec643504a8-kube-api-access-rm5dp\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.585691 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.585659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-workload-certs\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.585691 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.585702    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-envoy\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.585962 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.585722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm5dp\" (UniqueName: \"kubernetes.io/projected/acd5b194-6f4c-4cef-9342-95ec643504a8-kube-api-access-rm5dp\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.585962 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.585758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-data\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.585962 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.585869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-credential-socket\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.585962 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.585910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-token\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.585962 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.585957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-workload-socket\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.586264 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.585982    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-podinfo\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.586264 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.586011    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acd5b194-6f4c-4cef-9342-95ec643504a8-istiod-ca-cert\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.586264 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.586106    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-workload-certs\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.586264 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.586241    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-data\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.586560 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.586535    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-credential-socket\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.586560 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.586553    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-workload-socket\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.586916 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.586873    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acd5b194-6f4c-4cef-9342-95ec643504a8-istiod-ca-cert\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.588368 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.588345    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-envoy\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.588895 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.588872    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-podinfo\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.594169 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.594144    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-token\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.594275 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.594226    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm5dp\" (UniqueName: \"kubernetes.io/projected/acd5b194-6f4c-4cef-9342-95ec643504a8-kube-api-access-rm5dp\") pod \"e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") " pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.734888 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.734858    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:02.871586 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.871556    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"]
Jul 02 23:43:02.872987 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:43:02.872951    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd5b194_6f4c_4cef_9342_95ec643504a8.slice/crio-47278af6092da07ba782fdcac566f0896f7b7b63414cf97408ac2795aae7fba8 WatchSource:0}: Error finding container 47278af6092da07ba782fdcac566f0896f7b7b63414cf97408ac2795aae7fba8: Status 404 returned error can't find the container with id 47278af6092da07ba782fdcac566f0896f7b7b63414cf97408ac2795aae7fba8
Jul 02 23:43:02.875261 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.875228    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:43:02.875355 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.875296    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:43:02.875355 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:02.875329    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:43:03.599054 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:03.599012    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx" event={"ID":"acd5b194-6f4c-4cef-9342-95ec643504a8","Type":"ContainerStarted","Data":"d980b02f96a70bb25e74a3830e6ed9a246b71fdc651f761d5daff7fb2dbcbafd"}
Jul 02 23:43:03.599526 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:03.599079    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx" event={"ID":"acd5b194-6f4c-4cef-9342-95ec643504a8","Type":"ContainerStarted","Data":"47278af6092da07ba782fdcac566f0896f7b7b63414cf97408ac2795aae7fba8"}
Jul 02 23:43:03.620183 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:03.620138    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx" podStartSLOduration=1.62010423 podStartE2EDuration="1.62010423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:43:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:43:03.61869324 +0000 UTC m=+1928.872483241" watchObservedRunningTime="2026-07-02 23:43:03.62010423 +0000 UTC m=+1928.873894229"
Jul 02 23:43:03.735790 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:03.735750    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:03.743376 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:03.743344    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:04.604965 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:04.604933    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:04.605827 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:04.605808    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:12.269622 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.269578    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"]
Jul 02 23:43:12.270147 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.269936    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54158b19-17a7-4756-a060-61966d83d9ce" containerName="istio-proxy"
Jul 02 23:43:12.270147 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.269946    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="54158b19-17a7-4756-a060-61966d83d9ce" containerName="istio-proxy"
Jul 02 23:43:12.273258 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.273237    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.276288 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.276257    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9d43a8\"/\"openshift-service-ca.crt\""
Jul 02 23:43:12.277416 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.277397    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9d43a8\"/\"kube-root-ca.crt\""
Jul 02 23:43:12.277538 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.277483    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9d43a8\"/\"default-dockercfg-mlnrv\""
Jul 02 23:43:12.277538 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.277525    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9d43a8\"/\"auth-test-model-9d43a8-kserve-self-signed-certs\""
Jul 02 23:43:12.282419 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.282397    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"]
Jul 02 23:43:12.377624 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.377593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-home\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.377815 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.377644    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7nbh\" (UniqueName: \"kubernetes.io/projected/0471ea3d-ae78-4273-a770-119acb5e53a0-kube-api-access-f7nbh\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.377815 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.377689    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-model-cache\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.377815 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.377720    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0471ea3d-ae78-4273-a770-119acb5e53a0-tls-certs\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.377815 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.377744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-tmp-dir\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.377815 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.377766    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-dshm\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.479255 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.479218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-home\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.479454 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.479264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7nbh\" (UniqueName: \"kubernetes.io/projected/0471ea3d-ae78-4273-a770-119acb5e53a0-kube-api-access-f7nbh\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.479454 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.479293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-model-cache\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.479454 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.479314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0471ea3d-ae78-4273-a770-119acb5e53a0-tls-certs\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.479454 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.479333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-tmp-dir\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.479454 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.479347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-dshm\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.479796 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.479628    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-home\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.479796 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.479684    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-tmp-dir\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.479796 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.479716    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-model-cache\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.481534 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.481506    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-dshm\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.481967 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.481946    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0471ea3d-ae78-4273-a770-119acb5e53a0-tls-certs\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.487103 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.487084    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7nbh\" (UniqueName: \"kubernetes.io/projected/0471ea3d-ae78-4273-a770-119acb5e53a0-kube-api-access-f7nbh\") pod \"auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") " pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.584280 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.584188    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:12.721652 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:12.721627    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"]
Jul 02 23:43:12.723478 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:43:12.723451    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0471ea3d_ae78_4273_a770_119acb5e53a0.slice/crio-0f5cf205e16a6abe9538b01623558628370517754fe6aaa7d8dd571bfc4328dc WatchSource:0}: Error finding container 0f5cf205e16a6abe9538b01623558628370517754fe6aaa7d8dd571bfc4328dc: Status 404 returned error can't find the container with id 0f5cf205e16a6abe9538b01623558628370517754fe6aaa7d8dd571bfc4328dc
Jul 02 23:43:13.656015 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:13.655969    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47" event={"ID":"0471ea3d-ae78-4273-a770-119acb5e53a0","Type":"ContainerStarted","Data":"fad0fccd19e71af88d57b8128f78a85873fcfb7072671c570332d3bc2ca30bd8"}
Jul 02 23:43:13.656015 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:13.656018    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47" event={"ID":"0471ea3d-ae78-4273-a770-119acb5e53a0","Type":"ContainerStarted","Data":"0f5cf205e16a6abe9538b01623558628370517754fe6aaa7d8dd571bfc4328dc"}
Jul 02 23:43:13.673184 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:13.673132    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47" podStartSLOduration=1.6730983689999999 podStartE2EDuration="1.673098369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:43:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:43:13.67237342 +0000 UTC m=+1938.926163452" watchObservedRunningTime="2026-07-02 23:43:13.673098369 +0000 UTC m=+1938.926888390"
Jul 02 23:43:15.787105 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:15.787070    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:43:22.584326 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:22.584282    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:22.584326 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:22.584337    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:22.597327 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:22.597300    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:22.728682 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:22.728655    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:43:23.729048 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:23.729013    2580 generic.go:356] "Generic (PLEG): container finished" podID="acd5b194-6f4c-4cef-9342-95ec643504a8" containerID="d980b02f96a70bb25e74a3830e6ed9a246b71fdc651f761d5daff7fb2dbcbafd" exitCode=0
Jul 02 23:43:23.729592 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:23.729103    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx" event={"ID":"acd5b194-6f4c-4cef-9342-95ec643504a8","Type":"ContainerDied","Data":"d980b02f96a70bb25e74a3830e6ed9a246b71fdc651f761d5daff7fb2dbcbafd"}
Jul 02 23:43:23.729592 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:23.729538    2580 scope.go:117] "RemoveContainer" containerID="d980b02f96a70bb25e74a3830e6ed9a246b71fdc651f761d5daff7fb2dbcbafd"
Jul 02 23:43:23.730510 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:23.730480    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:43:23.730574 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:23.730560    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:43:23.730617 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:23.730602    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 02 23:43:24.739606 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:24.739559    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx" event={"ID":"acd5b194-6f4c-4cef-9342-95ec643504a8","Type":"ContainerStarted","Data":"16a2e71f1d15ce3ff9f6f018373fb47fa120f2a5f6efc009c5bcc7fc7cc91655"}
Jul 02 23:43:25.736197 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:25.736091    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:25.736584 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:25.736552    2580 patch_prober.go:28] interesting pod/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 02 23:43:25.736777 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:25.736751    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx" podUID="acd5b194-6f4c-4cef-9342-95ec643504a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 02 23:43:26.735605 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:26.735565    2580 patch_prober.go:28] interesting pod/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 02 23:43:26.736246 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:26.736209    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx" podUID="acd5b194-6f4c-4cef-9342-95ec643504a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 02 23:43:27.735794 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:27.735751    2580 patch_prober.go:28] interesting pod/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 02 23:43:27.736335 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:27.735829    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx" podUID="acd5b194-6f4c-4cef-9342-95ec643504a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 02 23:43:28.739045 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:28.739013    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:28.739500 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:28.739327    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:28.740017 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:28.740003    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:43:39.498678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.498648    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"]
Jul 02 23:43:39.502064 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.502045    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.505871 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.505844    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fcd282\"/\"openshift-service-ca.crt\""
Jul 02 23:43:39.505992 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.505843    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fcd282\"/\"auth-test-model-fcd282-kserve-self-signed-certs\""
Jul 02 23:43:39.505992 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.505981    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fcd282\"/\"default-dockercfg-nl8kc\""
Jul 02 23:43:39.506101 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.506025    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fcd282\"/\"kube-root-ca.crt\""
Jul 02 23:43:39.515562 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.515542    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"]
Jul 02 23:43:39.627098 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.627056    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp6wg\" (UniqueName: \"kubernetes.io/projected/8931570f-8b61-41ff-a8d2-817bba1b7de9-kube-api-access-dp6wg\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.627331 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.627175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-model-cache\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.627331 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.627208    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-dshm\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.627331 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.627222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8931570f-8b61-41ff-a8d2-817bba1b7de9-tls-certs\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.627500 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.627372    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-tmp-dir\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.627500 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.627413    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-home\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.728504 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.728468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp6wg\" (UniqueName: \"kubernetes.io/projected/8931570f-8b61-41ff-a8d2-817bba1b7de9-kube-api-access-dp6wg\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.728707 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.728537    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-model-cache\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.728707 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.728573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-dshm\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.728707 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.728598    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8931570f-8b61-41ff-a8d2-817bba1b7de9-tls-certs\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.728707 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.728681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-tmp-dir\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.728926 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.728788    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-home\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.729216 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.729187    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-model-cache\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.729358 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.729282    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-tmp-dir\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.729358 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.729285    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-home\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.731094 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.731065    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-dshm\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.731403 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.731385    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8931570f-8b61-41ff-a8d2-817bba1b7de9-tls-certs\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.738815 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.738791    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp6wg\" (UniqueName: \"kubernetes.io/projected/8931570f-8b61-41ff-a8d2-817bba1b7de9-kube-api-access-dp6wg\") pod \"auth-test-model-fcd282-kserve-f8ccb8559-z82mq\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") " pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.810810 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.810736    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:39.968098 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:39.968074    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"]
Jul 02 23:43:39.970441 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:43:39.970413    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8931570f_8b61_41ff_a8d2_817bba1b7de9.slice/crio-918db65faa849168889cb4d2fefd1b72bbabf330bafd585f0ed96efb70b67c68 WatchSource:0}: Error finding container 918db65faa849168889cb4d2fefd1b72bbabf330bafd585f0ed96efb70b67c68: Status 404 returned error can't find the container with id 918db65faa849168889cb4d2fefd1b72bbabf330bafd585f0ed96efb70b67c68
Jul 02 23:43:40.829599 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:40.829563    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq" event={"ID":"8931570f-8b61-41ff-a8d2-817bba1b7de9","Type":"ContainerStarted","Data":"2fde0d661c15cec8287d6e5ee750092238a5992145877f6ff0a3fda8a9d40d9f"}
Jul 02 23:43:40.830043 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:40.829608    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq" event={"ID":"8931570f-8b61-41ff-a8d2-817bba1b7de9","Type":"ContainerStarted","Data":"918db65faa849168889cb4d2fefd1b72bbabf330bafd585f0ed96efb70b67c68"}
Jul 02 23:43:40.849274 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:40.849220    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq" podStartSLOduration=1.8492023789999998 podStartE2EDuration="1.849202379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:43:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:43:40.847425744 +0000 UTC m=+1966.101215739" watchObservedRunningTime="2026-07-02 23:43:40.849202379 +0000 UTC m=+1966.102992377"
Jul 02 23:43:43.079312 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:43.079278    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:43:49.811380 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:49.811341    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:49.811380 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:49.811389    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:49.824079 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:49.824053    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:49.885450 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:49.885420    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:43:56.683266 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:43:56.683233    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:44:13.882662 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:44:13.882622    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:44:29.288083 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:44:29.288042    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:44:44.778169 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:44:44.778108    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:45:01.586062 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:01.586022    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:45:19.275847 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:19.275761    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:45:32.134028 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:32.133985    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"]
Jul 02 23:45:32.776015 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:32.775987    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"]
Jul 02 23:45:33.182806 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:33.182777    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:45:47.155006 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.154947    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47" podUID="0471ea3d-ae78-4273-a770-119acb5e53a0" containerName="main" containerID="cri-o://fad0fccd19e71af88d57b8128f78a85873fcfb7072671c570332d3bc2ca30bd8" gracePeriod=45
Jul 02 23:45:47.405356 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.405295    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:45:47.458378 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.458339    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0471ea3d-ae78-4273-a770-119acb5e53a0-tls-certs\") pod \"0471ea3d-ae78-4273-a770-119acb5e53a0\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") "
Jul 02 23:45:47.458532 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.458457    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-home\") pod \"0471ea3d-ae78-4273-a770-119acb5e53a0\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") "
Jul 02 23:45:47.458532 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.458483    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7nbh\" (UniqueName: \"kubernetes.io/projected/0471ea3d-ae78-4273-a770-119acb5e53a0-kube-api-access-f7nbh\") pod \"0471ea3d-ae78-4273-a770-119acb5e53a0\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") "
Jul 02 23:45:47.458532 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.458506    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-tmp-dir\") pod \"0471ea3d-ae78-4273-a770-119acb5e53a0\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") "
Jul 02 23:45:47.458674 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.458549    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-dshm\") pod \"0471ea3d-ae78-4273-a770-119acb5e53a0\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") "
Jul 02 23:45:47.458674 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.458649    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-model-cache\") pod \"0471ea3d-ae78-4273-a770-119acb5e53a0\" (UID: \"0471ea3d-ae78-4273-a770-119acb5e53a0\") "
Jul 02 23:45:47.458787 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.458733    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-home" (OuterVolumeSpecName: "home") pod "0471ea3d-ae78-4273-a770-119acb5e53a0" (UID: "0471ea3d-ae78-4273-a770-119acb5e53a0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:45:47.458843 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.458806    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0471ea3d-ae78-4273-a770-119acb5e53a0" (UID: "0471ea3d-ae78-4273-a770-119acb5e53a0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:45:47.458961 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.458943    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-model-cache" (OuterVolumeSpecName: "model-cache") pod "0471ea3d-ae78-4273-a770-119acb5e53a0" (UID: "0471ea3d-ae78-4273-a770-119acb5e53a0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:45:47.459003 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.458968    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-tmp-dir\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:45:47.459003 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.458982    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-home\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:45:47.460613 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.460590    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-dshm" (OuterVolumeSpecName: "dshm") pod "0471ea3d-ae78-4273-a770-119acb5e53a0" (UID: "0471ea3d-ae78-4273-a770-119acb5e53a0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:45:47.460821 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.460802    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0471ea3d-ae78-4273-a770-119acb5e53a0-kube-api-access-f7nbh" (OuterVolumeSpecName: "kube-api-access-f7nbh") pod "0471ea3d-ae78-4273-a770-119acb5e53a0" (UID: "0471ea3d-ae78-4273-a770-119acb5e53a0"). InnerVolumeSpecName "kube-api-access-f7nbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:45:47.460891 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.460806    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0471ea3d-ae78-4273-a770-119acb5e53a0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0471ea3d-ae78-4273-a770-119acb5e53a0" (UID: "0471ea3d-ae78-4273-a770-119acb5e53a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 23:45:47.486352 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.486320    2580 generic.go:356] "Generic (PLEG): container finished" podID="0471ea3d-ae78-4273-a770-119acb5e53a0" containerID="fad0fccd19e71af88d57b8128f78a85873fcfb7072671c570332d3bc2ca30bd8" exitCode=0
Jul 02 23:45:47.486466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.486373    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"
Jul 02 23:45:47.486466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.486405    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47" event={"ID":"0471ea3d-ae78-4273-a770-119acb5e53a0","Type":"ContainerDied","Data":"fad0fccd19e71af88d57b8128f78a85873fcfb7072671c570332d3bc2ca30bd8"}
Jul 02 23:45:47.486466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.486449    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47" event={"ID":"0471ea3d-ae78-4273-a770-119acb5e53a0","Type":"ContainerDied","Data":"0f5cf205e16a6abe9538b01623558628370517754fe6aaa7d8dd571bfc4328dc"}
Jul 02 23:45:47.486466 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.486466    2580 scope.go:117] "RemoveContainer" containerID="fad0fccd19e71af88d57b8128f78a85873fcfb7072671c570332d3bc2ca30bd8"
Jul 02 23:45:47.495529 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.495510    2580 scope.go:117] "RemoveContainer" containerID="fad0fccd19e71af88d57b8128f78a85873fcfb7072671c570332d3bc2ca30bd8"
Jul 02 23:45:47.495791 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:45:47.495774    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fad0fccd19e71af88d57b8128f78a85873fcfb7072671c570332d3bc2ca30bd8\": container with ID starting with fad0fccd19e71af88d57b8128f78a85873fcfb7072671c570332d3bc2ca30bd8 not found: ID does not exist" containerID="fad0fccd19e71af88d57b8128f78a85873fcfb7072671c570332d3bc2ca30bd8"
Jul 02 23:45:47.495839 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.495800    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fad0fccd19e71af88d57b8128f78a85873fcfb7072671c570332d3bc2ca30bd8"} err="failed to get container status \"fad0fccd19e71af88d57b8128f78a85873fcfb7072671c570332d3bc2ca30bd8\": rpc error: code = NotFound desc = could not find container \"fad0fccd19e71af88d57b8128f78a85873fcfb7072671c570332d3bc2ca30bd8\": container with ID starting with fad0fccd19e71af88d57b8128f78a85873fcfb7072671c570332d3bc2ca30bd8 not found: ID does not exist"
Jul 02 23:45:47.509072 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.509049    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"]
Jul 02 23:45:47.513352 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.513332    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9d43a8/auth-test-model-9d43a8-kserve-5fcf6d5f87-dsp47"]
Jul 02 23:45:47.559876 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.559847    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-dshm\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:45:47.559876 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.559872    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0471ea3d-ae78-4273-a770-119acb5e53a0-model-cache\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:45:47.559876 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.559882    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0471ea3d-ae78-4273-a770-119acb5e53a0-tls-certs\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:45:47.560068 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.559890    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7nbh\" (UniqueName: \"kubernetes.io/projected/0471ea3d-ae78-4273-a770-119acb5e53a0-kube-api-access-f7nbh\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:45:47.795891 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:47.795839    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq" podUID="8931570f-8b61-41ff-a8d2-817bba1b7de9" containerName="main" containerID="cri-o://2fde0d661c15cec8287d6e5ee750092238a5992145877f6ff0a3fda8a9d40d9f" gracePeriod=45
Jul 02 23:45:48.037323 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.037294    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:45:48.164585 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.164509    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-tmp-dir\") pod \"8931570f-8b61-41ff-a8d2-817bba1b7de9\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") "
Jul 02 23:45:48.164585 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.164568    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8931570f-8b61-41ff-a8d2-817bba1b7de9-tls-certs\") pod \"8931570f-8b61-41ff-a8d2-817bba1b7de9\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") "
Jul 02 23:45:48.165067 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.164594    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-model-cache\") pod \"8931570f-8b61-41ff-a8d2-817bba1b7de9\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") "
Jul 02 23:45:48.165067 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.164621    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-home\") pod \"8931570f-8b61-41ff-a8d2-817bba1b7de9\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") "
Jul 02 23:45:48.165067 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.164700    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-dshm\") pod \"8931570f-8b61-41ff-a8d2-817bba1b7de9\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") "
Jul 02 23:45:48.165067 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.164717    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8931570f-8b61-41ff-a8d2-817bba1b7de9" (UID: "8931570f-8b61-41ff-a8d2-817bba1b7de9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:45:48.165067 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.164729    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp6wg\" (UniqueName: \"kubernetes.io/projected/8931570f-8b61-41ff-a8d2-817bba1b7de9-kube-api-access-dp6wg\") pod \"8931570f-8b61-41ff-a8d2-817bba1b7de9\" (UID: \"8931570f-8b61-41ff-a8d2-817bba1b7de9\") "
Jul 02 23:45:48.165067 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.164875    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-model-cache" (OuterVolumeSpecName: "model-cache") pod "8931570f-8b61-41ff-a8d2-817bba1b7de9" (UID: "8931570f-8b61-41ff-a8d2-817bba1b7de9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:45:48.165067 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.164941    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-home" (OuterVolumeSpecName: "home") pod "8931570f-8b61-41ff-a8d2-817bba1b7de9" (UID: "8931570f-8b61-41ff-a8d2-817bba1b7de9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:45:48.165471 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.165099    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-home\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:45:48.165471 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.165133    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-tmp-dir\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:45:48.165471 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.165145    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-model-cache\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:45:48.167051 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.166696    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8931570f-8b61-41ff-a8d2-817bba1b7de9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8931570f-8b61-41ff-a8d2-817bba1b7de9" (UID: "8931570f-8b61-41ff-a8d2-817bba1b7de9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 23:45:48.167242 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.167112    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-dshm" (OuterVolumeSpecName: "dshm") pod "8931570f-8b61-41ff-a8d2-817bba1b7de9" (UID: "8931570f-8b61-41ff-a8d2-817bba1b7de9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:45:48.167242 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.167170    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8931570f-8b61-41ff-a8d2-817bba1b7de9-kube-api-access-dp6wg" (OuterVolumeSpecName: "kube-api-access-dp6wg") pod "8931570f-8b61-41ff-a8d2-817bba1b7de9" (UID: "8931570f-8b61-41ff-a8d2-817bba1b7de9"). InnerVolumeSpecName "kube-api-access-dp6wg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:45:48.265916 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.265881    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8931570f-8b61-41ff-a8d2-817bba1b7de9-dshm\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:45:48.265916 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.265912    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dp6wg\" (UniqueName: \"kubernetes.io/projected/8931570f-8b61-41ff-a8d2-817bba1b7de9-kube-api-access-dp6wg\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:45:48.265916 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.265922    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8931570f-8b61-41ff-a8d2-817bba1b7de9-tls-certs\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:45:48.494319 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.494275    2580 generic.go:356] "Generic (PLEG): container finished" podID="8931570f-8b61-41ff-a8d2-817bba1b7de9" containerID="2fde0d661c15cec8287d6e5ee750092238a5992145877f6ff0a3fda8a9d40d9f" exitCode=0
Jul 02 23:45:48.494319 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.494321    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"
Jul 02 23:45:48.494554 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.494341    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq" event={"ID":"8931570f-8b61-41ff-a8d2-817bba1b7de9","Type":"ContainerDied","Data":"2fde0d661c15cec8287d6e5ee750092238a5992145877f6ff0a3fda8a9d40d9f"}
Jul 02 23:45:48.494554 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.494380    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq" event={"ID":"8931570f-8b61-41ff-a8d2-817bba1b7de9","Type":"ContainerDied","Data":"918db65faa849168889cb4d2fefd1b72bbabf330bafd585f0ed96efb70b67c68"}
Jul 02 23:45:48.494554 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.494401    2580 scope.go:117] "RemoveContainer" containerID="2fde0d661c15cec8287d6e5ee750092238a5992145877f6ff0a3fda8a9d40d9f"
Jul 02 23:45:48.503884 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.503868    2580 scope.go:117] "RemoveContainer" containerID="2fde0d661c15cec8287d6e5ee750092238a5992145877f6ff0a3fda8a9d40d9f"
Jul 02 23:45:48.504168 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:45:48.504146    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fde0d661c15cec8287d6e5ee750092238a5992145877f6ff0a3fda8a9d40d9f\": container with ID starting with 2fde0d661c15cec8287d6e5ee750092238a5992145877f6ff0a3fda8a9d40d9f not found: ID does not exist" containerID="2fde0d661c15cec8287d6e5ee750092238a5992145877f6ff0a3fda8a9d40d9f"
Jul 02 23:45:48.504253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.504181    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fde0d661c15cec8287d6e5ee750092238a5992145877f6ff0a3fda8a9d40d9f"} err="failed to get container status \"2fde0d661c15cec8287d6e5ee750092238a5992145877f6ff0a3fda8a9d40d9f\": rpc error: code = NotFound desc = could not find container \"2fde0d661c15cec8287d6e5ee750092238a5992145877f6ff0a3fda8a9d40d9f\": container with ID starting with 2fde0d661c15cec8287d6e5ee750092238a5992145877f6ff0a3fda8a9d40d9f not found: ID does not exist"
Jul 02 23:45:48.516931 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.516905    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"]
Jul 02 23:45:48.520100 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:48.520078    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/auth-test-model-fcd282-kserve-f8ccb8559-z82mq"]
Jul 02 23:45:49.079448 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:49.079416    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:45:49.283283 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:49.283250    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0471ea3d-ae78-4273-a770-119acb5e53a0" path="/var/lib/kubelet/pods/0471ea3d-ae78-4273-a770-119acb5e53a0/volumes"
Jul 02 23:45:49.283649 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:45:49.283622    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8931570f-8b61-41ff-a8d2-817bba1b7de9" path="/var/lib/kubelet/pods/8931570f-8b61-41ff-a8d2-817bba1b7de9/volumes"
Jul 02 23:46:18.493505 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.493468    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"]
Jul 02 23:46:18.493980 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.493825    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8931570f-8b61-41ff-a8d2-817bba1b7de9" containerName="main"
Jul 02 23:46:18.493980 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.493837    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8931570f-8b61-41ff-a8d2-817bba1b7de9" containerName="main"
Jul 02 23:46:18.493980 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.493861    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0471ea3d-ae78-4273-a770-119acb5e53a0" containerName="main"
Jul 02 23:46:18.493980 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.493866    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0471ea3d-ae78-4273-a770-119acb5e53a0" containerName="main"
Jul 02 23:46:18.493980 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.493943    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0471ea3d-ae78-4273-a770-119acb5e53a0" containerName="main"
Jul 02 23:46:18.493980 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.493952    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8931570f-8b61-41ff-a8d2-817bba1b7de9" containerName="main"
Jul 02 23:46:18.496920 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.496897    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.500735 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.500713    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fcd282\"/\"openshift-service-ca.crt\""
Jul 02 23:46:18.500852 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.500713    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fcd282\"/\"default-dockercfg-nl8kc\""
Jul 02 23:46:18.500852 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.500750    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fcd282\"/\"kube-root-ca.crt\""
Jul 02 23:46:18.500852 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.500757    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fcd282\"/\"sub-test-model-fcd282-kserve-self-signed-certs\""
Jul 02 23:46:18.505142 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.505106    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"]
Jul 02 23:46:18.641188 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.641156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-tmp-dir\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.641381 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.641196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hls9\" (UniqueName: \"kubernetes.io/projected/e33df13a-ecac-4932-b826-a93b9eb20762-kube-api-access-9hls9\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.641381 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.641271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-model-cache\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.641381 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.641314    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-dshm\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.641381 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.641358    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-home\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.641570 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.641395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e33df13a-ecac-4932-b826-a93b9eb20762-tls-certs\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.742801 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.742769    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-tmp-dir\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.742982 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.742815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hls9\" (UniqueName: \"kubernetes.io/projected/e33df13a-ecac-4932-b826-a93b9eb20762-kube-api-access-9hls9\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.742982 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.742862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-model-cache\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.742982 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.742908    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-dshm\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.742982 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.742941    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-home\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.742982 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.742972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e33df13a-ecac-4932-b826-a93b9eb20762-tls-certs\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.743552 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.743487    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-tmp-dir\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.743736 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.743701    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-model-cache\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.743848 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.743759    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-home\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.747221 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.747198    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e33df13a-ecac-4932-b826-a93b9eb20762-tls-certs\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.753496 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.753473    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-dshm\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.753599 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.753507    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hls9\" (UniqueName: \"kubernetes.io/projected/e33df13a-ecac-4932-b826-a93b9eb20762-kube-api-access-9hls9\") pod \"sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") " pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.807346 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.807301    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:18.936749 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:18.936714    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"]
Jul 02 23:46:18.940054 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:46:18.940018    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode33df13a_ecac_4932_b826_a93b9eb20762.slice/crio-4e24cb7b65c68c41b73af5d0e9626b910367ac4d6e25efcc1d28f0f05318324d WatchSource:0}: Error finding container 4e24cb7b65c68c41b73af5d0e9626b910367ac4d6e25efcc1d28f0f05318324d: Status 404 returned error can't find the container with id 4e24cb7b65c68c41b73af5d0e9626b910367ac4d6e25efcc1d28f0f05318324d
Jul 02 23:46:19.652143 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:19.652092    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q" event={"ID":"e33df13a-ecac-4932-b826-a93b9eb20762","Type":"ContainerStarted","Data":"ae0a8e9e9d48ac654efc1c1436eb969e387f24db222561be715fa8d9379dcb73"}
Jul 02 23:46:19.652541 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:19.652152    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q" event={"ID":"e33df13a-ecac-4932-b826-a93b9eb20762","Type":"ContainerStarted","Data":"4e24cb7b65c68c41b73af5d0e9626b910367ac4d6e25efcc1d28f0f05318324d"}
Jul 02 23:46:19.687824 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:19.687773    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q" podStartSLOduration=1.6877554209999999 podStartE2EDuration="1.687755421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:46:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:46:19.680950538 +0000 UTC m=+2124.934740538" watchObservedRunningTime="2026-07-02 23:46:19.687755421 +0000 UTC m=+2124.941545419"
Jul 02 23:46:23.090707 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:23.090671    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:46:28.807761 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:28.807721    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:28.808308 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:28.807792    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:28.820335 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:28.820313    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:29.718564 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:29.718533    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:46:45.272760 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:46:45.272726    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:47:02.396589 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:02.396554    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"]
Jul 02 23:47:05.472362 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:05.472328    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:47:15.698408 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.698365    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"]
Jul 02 23:47:15.701880 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.701850    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.704890 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.704870    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9d43a8\"/\"kube-root-ca.crt\""
Jul 02 23:47:15.706255 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.706233    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9d43a8\"/\"default-dockercfg-mlnrv\""
Jul 02 23:47:15.706367 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.706257    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9d43a8\"/\"openshift-service-ca.crt\""
Jul 02 23:47:15.706367 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.706257    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9d43a8\"/\"rate-test-model-9d43a8-kserve-self-signed-certs\""
Jul 02 23:47:15.710607 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.710580    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"]
Jul 02 23:47:15.744656 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.744615    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-home\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.744656 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.744654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-dshm\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.744869 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.744742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l6nn\" (UniqueName: \"kubernetes.io/projected/654acc03-ac32-484a-a540-f6d1807efee5-kube-api-access-8l6nn\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.744869 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.744798    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-tmp-dir\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.744869 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.744839    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/654acc03-ac32-484a-a540-f6d1807efee5-tls-certs\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.744869 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.744856    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-model-cache\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.845864 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.845821    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l6nn\" (UniqueName: \"kubernetes.io/projected/654acc03-ac32-484a-a540-f6d1807efee5-kube-api-access-8l6nn\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.846079 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.845895    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-tmp-dir\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.846079 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.845939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/654acc03-ac32-484a-a540-f6d1807efee5-tls-certs\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.846079 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.845962    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-model-cache\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.846079 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.846034    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-home\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.846079 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.846054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-dshm\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.846443 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.846418    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-tmp-dir\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.846509 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.846430    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-home\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.846509 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.846488    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-model-cache\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.848364 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.848340    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-dshm\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.848528 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.848511    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/654acc03-ac32-484a-a540-f6d1807efee5-tls-certs\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:15.854659 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:15.854638    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l6nn\" (UniqueName: \"kubernetes.io/projected/654acc03-ac32-484a-a540-f6d1807efee5-kube-api-access-8l6nn\") pod \"rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") " pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:16.012049 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:16.011960    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:16.153232 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:16.153202    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"]
Jul 02 23:47:16.155026 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:47:16.155001    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod654acc03_ac32_484a_a540_f6d1807efee5.slice/crio-a82f14f9e1a01d5f00891716416c851feca711715d3c6a82be947ebd77baa9a4 WatchSource:0}: Error finding container a82f14f9e1a01d5f00891716416c851feca711715d3c6a82be947ebd77baa9a4: Status 404 returned error can't find the container with id a82f14f9e1a01d5f00891716416c851feca711715d3c6a82be947ebd77baa9a4
Jul 02 23:47:16.938630 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:16.938594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g" event={"ID":"654acc03-ac32-484a-a540-f6d1807efee5","Type":"ContainerStarted","Data":"01d84006f6e802be70f6596d2673c68ad824644c8ab0b4bf0604b1182acd7710"}
Jul 02 23:47:16.939097 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:16.938643    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g" event={"ID":"654acc03-ac32-484a-a540-f6d1807efee5","Type":"ContainerStarted","Data":"a82f14f9e1a01d5f00891716416c851feca711715d3c6a82be947ebd77baa9a4"}
Jul 02 23:47:16.955094 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:16.955049    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g" podStartSLOduration=1.955035052 podStartE2EDuration="1.955035052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:47:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:47:16.954594476 +0000 UTC m=+2182.208384476" watchObservedRunningTime="2026-07-02 23:47:16.955035052 +0000 UTC m=+2182.208825050"
Jul 02 23:47:17.416016 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.415931    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q" podUID="e33df13a-ecac-4932-b826-a93b9eb20762" containerName="main" containerID="cri-o://ae0a8e9e9d48ac654efc1c1436eb969e387f24db222561be715fa8d9379dcb73" gracePeriod=45
Jul 02 23:47:17.675226 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.675165    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:47:17.760383 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.760350    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-tmp-dir\") pod \"e33df13a-ecac-4932-b826-a93b9eb20762\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") "
Jul 02 23:47:17.760383 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.760380    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-dshm\") pod \"e33df13a-ecac-4932-b826-a93b9eb20762\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") "
Jul 02 23:47:17.760633 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.760424    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e33df13a-ecac-4932-b826-a93b9eb20762-tls-certs\") pod \"e33df13a-ecac-4932-b826-a93b9eb20762\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") "
Jul 02 23:47:17.760633 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.760473    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-model-cache\") pod \"e33df13a-ecac-4932-b826-a93b9eb20762\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") "
Jul 02 23:47:17.760633 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.760618    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-home\") pod \"e33df13a-ecac-4932-b826-a93b9eb20762\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") "
Jul 02 23:47:17.760805 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.760647    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e33df13a-ecac-4932-b826-a93b9eb20762" (UID: "e33df13a-ecac-4932-b826-a93b9eb20762"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:47:17.760805 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.760678    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hls9\" (UniqueName: \"kubernetes.io/projected/e33df13a-ecac-4932-b826-a93b9eb20762-kube-api-access-9hls9\") pod \"e33df13a-ecac-4932-b826-a93b9eb20762\" (UID: \"e33df13a-ecac-4932-b826-a93b9eb20762\") "
Jul 02 23:47:17.760805 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.760711    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-model-cache" (OuterVolumeSpecName: "model-cache") pod "e33df13a-ecac-4932-b826-a93b9eb20762" (UID: "e33df13a-ecac-4932-b826-a93b9eb20762"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:47:17.760975 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.760897    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-home" (OuterVolumeSpecName: "home") pod "e33df13a-ecac-4932-b826-a93b9eb20762" (UID: "e33df13a-ecac-4932-b826-a93b9eb20762"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:47:17.761104 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.761086    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-home\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:47:17.761183 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.761111    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-tmp-dir\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:47:17.761183 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.761160    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-model-cache\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:47:17.762441 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.762420    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-dshm" (OuterVolumeSpecName: "dshm") pod "e33df13a-ecac-4932-b826-a93b9eb20762" (UID: "e33df13a-ecac-4932-b826-a93b9eb20762"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:47:17.762536 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.762522    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e33df13a-ecac-4932-b826-a93b9eb20762-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e33df13a-ecac-4932-b826-a93b9eb20762" (UID: "e33df13a-ecac-4932-b826-a93b9eb20762"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 23:47:17.762613 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.762593    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e33df13a-ecac-4932-b826-a93b9eb20762-kube-api-access-9hls9" (OuterVolumeSpecName: "kube-api-access-9hls9") pod "e33df13a-ecac-4932-b826-a93b9eb20762" (UID: "e33df13a-ecac-4932-b826-a93b9eb20762"). InnerVolumeSpecName "kube-api-access-9hls9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:47:17.862554 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.862516    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e33df13a-ecac-4932-b826-a93b9eb20762-tls-certs\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:47:17.862554 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.862551    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hls9\" (UniqueName: \"kubernetes.io/projected/e33df13a-ecac-4932-b826-a93b9eb20762-kube-api-access-9hls9\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:47:17.862554 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.862561    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e33df13a-ecac-4932-b826-a93b9eb20762-dshm\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:47:17.945190 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.945149    2580 generic.go:356] "Generic (PLEG): container finished" podID="e33df13a-ecac-4932-b826-a93b9eb20762" containerID="ae0a8e9e9d48ac654efc1c1436eb969e387f24db222561be715fa8d9379dcb73" exitCode=0
Jul 02 23:47:17.945646 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.945249    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"
Jul 02 23:47:17.945646 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.945287    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q" event={"ID":"e33df13a-ecac-4932-b826-a93b9eb20762","Type":"ContainerDied","Data":"ae0a8e9e9d48ac654efc1c1436eb969e387f24db222561be715fa8d9379dcb73"}
Jul 02 23:47:17.945646 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.945327    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q" event={"ID":"e33df13a-ecac-4932-b826-a93b9eb20762","Type":"ContainerDied","Data":"4e24cb7b65c68c41b73af5d0e9626b910367ac4d6e25efcc1d28f0f05318324d"}
Jul 02 23:47:17.945646 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.945349    2580 scope.go:117] "RemoveContainer" containerID="ae0a8e9e9d48ac654efc1c1436eb969e387f24db222561be715fa8d9379dcb73"
Jul 02 23:47:17.954925 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.954907    2580 scope.go:117] "RemoveContainer" containerID="ae0a8e9e9d48ac654efc1c1436eb969e387f24db222561be715fa8d9379dcb73"
Jul 02 23:47:17.955241 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:47:17.955213    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae0a8e9e9d48ac654efc1c1436eb969e387f24db222561be715fa8d9379dcb73\": container with ID starting with ae0a8e9e9d48ac654efc1c1436eb969e387f24db222561be715fa8d9379dcb73 not found: ID does not exist" containerID="ae0a8e9e9d48ac654efc1c1436eb969e387f24db222561be715fa8d9379dcb73"
Jul 02 23:47:17.955299 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.955252    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae0a8e9e9d48ac654efc1c1436eb969e387f24db222561be715fa8d9379dcb73"} err="failed to get container status \"ae0a8e9e9d48ac654efc1c1436eb969e387f24db222561be715fa8d9379dcb73\": rpc error: code = NotFound desc = could not find container \"ae0a8e9e9d48ac654efc1c1436eb969e387f24db222561be715fa8d9379dcb73\": container with ID starting with ae0a8e9e9d48ac654efc1c1436eb969e387f24db222561be715fa8d9379dcb73 not found: ID does not exist"
Jul 02 23:47:17.967848 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.967824    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"]
Jul 02 23:47:17.971588 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:17.971566    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/sub-test-model-fcd282-kserve-fc7f7d58f-6hd9q"]
Jul 02 23:47:18.882914 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:18.882875    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:47:19.283225 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:19.283195    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e33df13a-ecac-4932-b826-a93b9eb20762" path="/var/lib/kubelet/pods/e33df13a-ecac-4932-b826-a93b9eb20762/volumes"
Jul 02 23:47:26.012960 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:26.012917    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:26.013474 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:26.012977    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:26.025507 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:26.025480    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:47:27.001475 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:47:27.001447    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:48:02.289345 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.289305    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"]
Jul 02 23:48:02.289816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.289671    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e33df13a-ecac-4932-b826-a93b9eb20762" containerName="main"
Jul 02 23:48:02.289816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.289685    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e33df13a-ecac-4932-b826-a93b9eb20762" containerName="main"
Jul 02 23:48:02.289816 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.289797    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e33df13a-ecac-4932-b826-a93b9eb20762" containerName="main"
Jul 02 23:48:02.292789 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.292769    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.296791 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.296762    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fcd282\"/\"openshift-service-ca.crt\""
Jul 02 23:48:02.296937 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.296792    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fcd282\"/\"kube-root-ca.crt\""
Jul 02 23:48:02.296937 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.296764    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fcd282\"/\"default-dockercfg-nl8kc\""
Jul 02 23:48:02.296937 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.296840    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fcd282\"/\"rate-test-model-fcd282-kserve-self-signed-certs\""
Jul 02 23:48:02.303295 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.303272    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"]
Jul 02 23:48:02.369445 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.369403    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-tmp-dir\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.369445 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.369446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prcz2\" (UniqueName: \"kubernetes.io/projected/a45c5458-50c5-4662-aaff-58837fe2c2e1-kube-api-access-prcz2\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.369702 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.369580    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-dshm\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.369702 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.369638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-home\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.369702 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.369672    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a45c5458-50c5-4662-aaff-58837fe2c2e1-tls-certs\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.369848 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.369745    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-model-cache\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.471144 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.471071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-tmp-dir\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.471144 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.471130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prcz2\" (UniqueName: \"kubernetes.io/projected/a45c5458-50c5-4662-aaff-58837fe2c2e1-kube-api-access-prcz2\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.471417 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.471185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-dshm\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.471417 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.471218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-home\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.471417 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.471244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a45c5458-50c5-4662-aaff-58837fe2c2e1-tls-certs\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.471417 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.471285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-model-cache\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.471615 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.471525    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-tmp-dir\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.471659 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.471643    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-model-cache\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.471694 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.471664    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-home\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.473452 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.473427    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-dshm\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.473765 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.473744    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a45c5458-50c5-4662-aaff-58837fe2c2e1-tls-certs\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.480140 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.480096    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prcz2\" (UniqueName: \"kubernetes.io/projected/a45c5458-50c5-4662-aaff-58837fe2c2e1-kube-api-access-prcz2\") pod \"rate-test-model-fcd282-kserve-cc5d595f5-s67qm\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") " pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.603192 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.603061    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:02.743161 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:02.743135    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"]
Jul 02 23:48:02.745360 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:48:02.745323    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda45c5458_50c5_4662_aaff_58837fe2c2e1.slice/crio-b20efed01882900e8ceb41cb72bb73efbe9d563c5f5a5cf8247e8a2ac8d0b175 WatchSource:0}: Error finding container b20efed01882900e8ceb41cb72bb73efbe9d563c5f5a5cf8247e8a2ac8d0b175: Status 404 returned error can't find the container with id b20efed01882900e8ceb41cb72bb73efbe9d563c5f5a5cf8247e8a2ac8d0b175
Jul 02 23:48:03.176138 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:03.176083    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm" event={"ID":"a45c5458-50c5-4662-aaff-58837fe2c2e1","Type":"ContainerStarted","Data":"c43d68eb280bc22c23387412b7577f00fba96947a8db5a918caf098f194d85b4"}
Jul 02 23:48:03.176326 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:03.176147    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm" event={"ID":"a45c5458-50c5-4662-aaff-58837fe2c2e1","Type":"ContainerStarted","Data":"b20efed01882900e8ceb41cb72bb73efbe9d563c5f5a5cf8247e8a2ac8d0b175"}
Jul 02 23:48:03.194297 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:03.194246    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm" podStartSLOduration=1.194227645 podStartE2EDuration="1.194227645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 23:48:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:48:03.192322737 +0000 UTC m=+2228.446112775" watchObservedRunningTime="2026-07-02 23:48:03.194227645 +0000 UTC m=+2228.448017643"
Jul 02 23:48:06.480316 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:06.480281    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:48:12.603841 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:12.603783    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:12.604356 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:12.604063    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:12.616494 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:12.616466    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:13.237168 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:13.237139    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:48:35.080801 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:35.080769    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:48:49.482675 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:49.482632    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:48:55.253019 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:55.252987    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"]
Jul 02 23:48:56.078425 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:48:56.078392    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"]
Jul 02 23:49:02.772712 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:02.772678    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:49:10.273293 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.273232    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g" podUID="654acc03-ac32-484a-a540-f6d1807efee5" containerName="main" containerID="cri-o://01d84006f6e802be70f6596d2673c68ad824644c8ab0b4bf0604b1182acd7710" gracePeriod=45
Jul 02 23:49:10.520678 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.520650    2580 generic.go:356] "Generic (PLEG): container finished" podID="654acc03-ac32-484a-a540-f6d1807efee5" containerID="01d84006f6e802be70f6596d2673c68ad824644c8ab0b4bf0604b1182acd7710" exitCode=0
Jul 02 23:49:10.520804 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.520699    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g" event={"ID":"654acc03-ac32-484a-a540-f6d1807efee5","Type":"ContainerDied","Data":"01d84006f6e802be70f6596d2673c68ad824644c8ab0b4bf0604b1182acd7710"}
Jul 02 23:49:10.520804 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.520727    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g" event={"ID":"654acc03-ac32-484a-a540-f6d1807efee5","Type":"ContainerDied","Data":"a82f14f9e1a01d5f00891716416c851feca711715d3c6a82be947ebd77baa9a4"}
Jul 02 23:49:10.520804 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.520738    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a82f14f9e1a01d5f00891716416c851feca711715d3c6a82be947ebd77baa9a4"
Jul 02 23:49:10.530681 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.530620    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:49:10.575601 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.575566    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-home\") pod \"654acc03-ac32-484a-a540-f6d1807efee5\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") "
Jul 02 23:49:10.575785 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.575614    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-dshm\") pod \"654acc03-ac32-484a-a540-f6d1807efee5\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") "
Jul 02 23:49:10.575785 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.575642    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/654acc03-ac32-484a-a540-f6d1807efee5-tls-certs\") pod \"654acc03-ac32-484a-a540-f6d1807efee5\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") "
Jul 02 23:49:10.575785 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.575664    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-model-cache\") pod \"654acc03-ac32-484a-a540-f6d1807efee5\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") "
Jul 02 23:49:10.575785 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.575714    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l6nn\" (UniqueName: \"kubernetes.io/projected/654acc03-ac32-484a-a540-f6d1807efee5-kube-api-access-8l6nn\") pod \"654acc03-ac32-484a-a540-f6d1807efee5\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") "
Jul 02 23:49:10.576016 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.575787    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-tmp-dir\") pod \"654acc03-ac32-484a-a540-f6d1807efee5\" (UID: \"654acc03-ac32-484a-a540-f6d1807efee5\") "
Jul 02 23:49:10.576016 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.575921    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-home" (OuterVolumeSpecName: "home") pod "654acc03-ac32-484a-a540-f6d1807efee5" (UID: "654acc03-ac32-484a-a540-f6d1807efee5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:49:10.576016 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.576001    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-model-cache" (OuterVolumeSpecName: "model-cache") pod "654acc03-ac32-484a-a540-f6d1807efee5" (UID: "654acc03-ac32-484a-a540-f6d1807efee5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:49:10.576198 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.576163    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-home\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:10.576198 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.576179    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "654acc03-ac32-484a-a540-f6d1807efee5" (UID: "654acc03-ac32-484a-a540-f6d1807efee5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:49:10.576312 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.576183    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-model-cache\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:10.577913 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.577881    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/654acc03-ac32-484a-a540-f6d1807efee5-kube-api-access-8l6nn" (OuterVolumeSpecName: "kube-api-access-8l6nn") pod "654acc03-ac32-484a-a540-f6d1807efee5" (UID: "654acc03-ac32-484a-a540-f6d1807efee5"). InnerVolumeSpecName "kube-api-access-8l6nn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:49:10.577913 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.577899    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/654acc03-ac32-484a-a540-f6d1807efee5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "654acc03-ac32-484a-a540-f6d1807efee5" (UID: "654acc03-ac32-484a-a540-f6d1807efee5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 23:49:10.578052 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.577894    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-dshm" (OuterVolumeSpecName: "dshm") pod "654acc03-ac32-484a-a540-f6d1807efee5" (UID: "654acc03-ac32-484a-a540-f6d1807efee5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:49:10.676762 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.676724    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l6nn\" (UniqueName: \"kubernetes.io/projected/654acc03-ac32-484a-a540-f6d1807efee5-kube-api-access-8l6nn\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:10.676762 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.676754    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-tmp-dir\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:10.676762 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.676763    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/654acc03-ac32-484a-a540-f6d1807efee5-dshm\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:10.677001 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:10.676773    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/654acc03-ac32-484a-a540-f6d1807efee5-tls-certs\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:11.098310 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.098262    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm" podUID="a45c5458-50c5-4662-aaff-58837fe2c2e1" containerName="main" containerID="cri-o://c43d68eb280bc22c23387412b7577f00fba96947a8db5a918caf098f194d85b4" gracePeriod=45
Jul 02 23:49:11.358032 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.357970    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:49:11.484866 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.484832    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-dshm\") pod \"a45c5458-50c5-4662-aaff-58837fe2c2e1\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") "
Jul 02 23:49:11.484866 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.484873    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a45c5458-50c5-4662-aaff-58837fe2c2e1-tls-certs\") pod \"a45c5458-50c5-4662-aaff-58837fe2c2e1\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") "
Jul 02 23:49:11.485153 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.484952    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-tmp-dir\") pod \"a45c5458-50c5-4662-aaff-58837fe2c2e1\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") "
Jul 02 23:49:11.485153 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.484974    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-home\") pod \"a45c5458-50c5-4662-aaff-58837fe2c2e1\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") "
Jul 02 23:49:11.485153 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.484993    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prcz2\" (UniqueName: \"kubernetes.io/projected/a45c5458-50c5-4662-aaff-58837fe2c2e1-kube-api-access-prcz2\") pod \"a45c5458-50c5-4662-aaff-58837fe2c2e1\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") "
Jul 02 23:49:11.485153 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.485011    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-model-cache\") pod \"a45c5458-50c5-4662-aaff-58837fe2c2e1\" (UID: \"a45c5458-50c5-4662-aaff-58837fe2c2e1\") "
Jul 02 23:49:11.485373 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.485292    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-home" (OuterVolumeSpecName: "home") pod "a45c5458-50c5-4662-aaff-58837fe2c2e1" (UID: "a45c5458-50c5-4662-aaff-58837fe2c2e1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:49:11.485373 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.485323    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a45c5458-50c5-4662-aaff-58837fe2c2e1" (UID: "a45c5458-50c5-4662-aaff-58837fe2c2e1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:49:11.485373 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.485356    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-model-cache" (OuterVolumeSpecName: "model-cache") pod "a45c5458-50c5-4662-aaff-58837fe2c2e1" (UID: "a45c5458-50c5-4662-aaff-58837fe2c2e1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:49:11.486985 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.486962    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-dshm" (OuterVolumeSpecName: "dshm") pod "a45c5458-50c5-4662-aaff-58837fe2c2e1" (UID: "a45c5458-50c5-4662-aaff-58837fe2c2e1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:49:11.487094 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.487067    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a45c5458-50c5-4662-aaff-58837fe2c2e1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a45c5458-50c5-4662-aaff-58837fe2c2e1" (UID: "a45c5458-50c5-4662-aaff-58837fe2c2e1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 02 23:49:11.487165 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.487112    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a45c5458-50c5-4662-aaff-58837fe2c2e1-kube-api-access-prcz2" (OuterVolumeSpecName: "kube-api-access-prcz2") pod "a45c5458-50c5-4662-aaff-58837fe2c2e1" (UID: "a45c5458-50c5-4662-aaff-58837fe2c2e1"). InnerVolumeSpecName "kube-api-access-prcz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:49:11.532751 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.532719    2580 generic.go:356] "Generic (PLEG): container finished" podID="a45c5458-50c5-4662-aaff-58837fe2c2e1" containerID="c43d68eb280bc22c23387412b7577f00fba96947a8db5a918caf098f194d85b4" exitCode=0
Jul 02 23:49:11.532917 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.532772    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"
Jul 02 23:49:11.532917 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.532790    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"
Jul 02 23:49:11.532917 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.532809    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm" event={"ID":"a45c5458-50c5-4662-aaff-58837fe2c2e1","Type":"ContainerDied","Data":"c43d68eb280bc22c23387412b7577f00fba96947a8db5a918caf098f194d85b4"}
Jul 02 23:49:11.532917 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.532852    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm" event={"ID":"a45c5458-50c5-4662-aaff-58837fe2c2e1","Type":"ContainerDied","Data":"b20efed01882900e8ceb41cb72bb73efbe9d563c5f5a5cf8247e8a2ac8d0b175"}
Jul 02 23:49:11.532917 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.532870    2580 scope.go:117] "RemoveContainer" containerID="c43d68eb280bc22c23387412b7577f00fba96947a8db5a918caf098f194d85b4"
Jul 02 23:49:11.542000 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.541979    2580 scope.go:117] "RemoveContainer" containerID="c43d68eb280bc22c23387412b7577f00fba96947a8db5a918caf098f194d85b4"
Jul 02 23:49:11.542307 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:49:11.542289    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c43d68eb280bc22c23387412b7577f00fba96947a8db5a918caf098f194d85b4\": container with ID starting with c43d68eb280bc22c23387412b7577f00fba96947a8db5a918caf098f194d85b4 not found: ID does not exist" containerID="c43d68eb280bc22c23387412b7577f00fba96947a8db5a918caf098f194d85b4"
Jul 02 23:49:11.542369 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.542317    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c43d68eb280bc22c23387412b7577f00fba96947a8db5a918caf098f194d85b4"} err="failed to get container status \"c43d68eb280bc22c23387412b7577f00fba96947a8db5a918caf098f194d85b4\": rpc error: code = NotFound desc = could not find container \"c43d68eb280bc22c23387412b7577f00fba96947a8db5a918caf098f194d85b4\": container with ID starting with c43d68eb280bc22c23387412b7577f00fba96947a8db5a918caf098f194d85b4 not found: ID does not exist"
Jul 02 23:49:11.552014 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.551992    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"]
Jul 02 23:49:11.557207 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.557187    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9d43a8/rate-test-model-9d43a8-kserve-677b99b5ff-4nt8g"]
Jul 02 23:49:11.570082 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.570053    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"]
Jul 02 23:49:11.576314 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.576286    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-fcd282/rate-test-model-fcd282-kserve-cc5d595f5-s67qm"]
Jul 02 23:49:11.585938 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.585920    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-home\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:11.586038 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.585939    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prcz2\" (UniqueName: \"kubernetes.io/projected/a45c5458-50c5-4662-aaff-58837fe2c2e1-kube-api-access-prcz2\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:11.586038 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.585951    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-model-cache\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:11.586038 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.585961    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-dshm\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:11.586038 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.585968    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a45c5458-50c5-4662-aaff-58837fe2c2e1-tls-certs\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:11.586038 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:11.585994    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a45c5458-50c5-4662-aaff-58837fe2c2e1-tmp-dir\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:13.282582 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:13.282546    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="654acc03-ac32-484a-a540-f6d1807efee5" path="/var/lib/kubelet/pods/654acc03-ac32-484a-a540-f6d1807efee5/volumes"
Jul 02 23:49:13.282998 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:13.282931    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a45c5458-50c5-4662-aaff-58837fe2c2e1" path="/var/lib/kubelet/pods/a45c5458-50c5-4662-aaff-58837fe2c2e1/volumes"
Jul 02 23:49:43.988726 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:43.988688    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5czdx"]
Jul 02 23:49:46.916987 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:46.916953    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"]
Jul 02 23:49:46.917455 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:46.917252    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx" podUID="acd5b194-6f4c-4cef-9342-95ec643504a8" containerName="istio-proxy" containerID="cri-o://16a2e71f1d15ce3ff9f6f018373fb47fa120f2a5f6efc009c5bcc7fc7cc91655" gracePeriod=30
Jul 02 23:49:52.173617 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.173592    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:49:52.248841 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.248800    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-workload-socket\") pod \"acd5b194-6f4c-4cef-9342-95ec643504a8\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") "
Jul 02 23:49:52.248841 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.248839    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-data\") pod \"acd5b194-6f4c-4cef-9342-95ec643504a8\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") "
Jul 02 23:49:52.249112 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.248870    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acd5b194-6f4c-4cef-9342-95ec643504a8-istiod-ca-cert\") pod \"acd5b194-6f4c-4cef-9342-95ec643504a8\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") "
Jul 02 23:49:52.249112 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.248915    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rm5dp\" (UniqueName: \"kubernetes.io/projected/acd5b194-6f4c-4cef-9342-95ec643504a8-kube-api-access-rm5dp\") pod \"acd5b194-6f4c-4cef-9342-95ec643504a8\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") "
Jul 02 23:49:52.249112 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.248937    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-podinfo\") pod \"acd5b194-6f4c-4cef-9342-95ec643504a8\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") "
Jul 02 23:49:52.249112 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.248956    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-workload-certs\") pod \"acd5b194-6f4c-4cef-9342-95ec643504a8\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") "
Jul 02 23:49:52.249112 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.248994    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-envoy\") pod \"acd5b194-6f4c-4cef-9342-95ec643504a8\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") "
Jul 02 23:49:52.249112 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.249035    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-credential-socket\") pod \"acd5b194-6f4c-4cef-9342-95ec643504a8\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") "
Jul 02 23:49:52.249112 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.249068    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-token\") pod \"acd5b194-6f4c-4cef-9342-95ec643504a8\" (UID: \"acd5b194-6f4c-4cef-9342-95ec643504a8\") "
Jul 02 23:49:52.249485 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.249172    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "acd5b194-6f4c-4cef-9342-95ec643504a8" (UID: "acd5b194-6f4c-4cef-9342-95ec643504a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:49:52.249485 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.249314    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "acd5b194-6f4c-4cef-9342-95ec643504a8" (UID: "acd5b194-6f4c-4cef-9342-95ec643504a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:49:52.249485 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.249338    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-workload-socket\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:52.249485 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.249411    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "acd5b194-6f4c-4cef-9342-95ec643504a8" (UID: "acd5b194-6f4c-4cef-9342-95ec643504a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:49:52.249485 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.249334    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acd5b194-6f4c-4cef-9342-95ec643504a8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "acd5b194-6f4c-4cef-9342-95ec643504a8" (UID: "acd5b194-6f4c-4cef-9342-95ec643504a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 02 23:49:52.249861 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.249830    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-data" (OuterVolumeSpecName: "istio-data") pod "acd5b194-6f4c-4cef-9342-95ec643504a8" (UID: "acd5b194-6f4c-4cef-9342-95ec643504a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:49:52.251278 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.251251    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acd5b194-6f4c-4cef-9342-95ec643504a8-kube-api-access-rm5dp" (OuterVolumeSpecName: "kube-api-access-rm5dp") pod "acd5b194-6f4c-4cef-9342-95ec643504a8" (UID: "acd5b194-6f4c-4cef-9342-95ec643504a8"). InnerVolumeSpecName "kube-api-access-rm5dp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:49:52.251391 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.251281    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "acd5b194-6f4c-4cef-9342-95ec643504a8" (UID: "acd5b194-6f4c-4cef-9342-95ec643504a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 02 23:49:52.251391 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.251296    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-token" (OuterVolumeSpecName: "istio-token") pod "acd5b194-6f4c-4cef-9342-95ec643504a8" (UID: "acd5b194-6f4c-4cef-9342-95ec643504a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 02 23:49:52.251391 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.251376    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "acd5b194-6f4c-4cef-9342-95ec643504a8" (UID: "acd5b194-6f4c-4cef-9342-95ec643504a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 02 23:49:52.267448 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.267420    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-w2pkr_a00b6be7-7525-4621-b309-07f433f73161/manager/0.log"
Jul 02 23:49:52.350485 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.350459    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-token\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:52.350485 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.350480    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-data\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:52.350485 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.350488    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acd5b194-6f4c-4cef-9342-95ec643504a8-istiod-ca-cert\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:52.350485 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.350496    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rm5dp\" (UniqueName: \"kubernetes.io/projected/acd5b194-6f4c-4cef-9342-95ec643504a8-kube-api-access-rm5dp\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:52.350783 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.350505    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-podinfo\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:52.350783 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.350516    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-workload-certs\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:52.350783 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.350527    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-istio-envoy\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:52.350783 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.350540    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd5b194-6f4c-4cef-9342-95ec643504a8-credential-socket\") on node \"ip-10-0-140-52.ec2.internal\" DevicePath \"\""
Jul 02 23:49:52.389844 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.389804    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5tvj9_3dbab3c8-c572-4831-b03e-1a94243d20b3/manager/0.log"
Jul 02 23:49:52.743370 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.743337    2580 generic.go:356] "Generic (PLEG): container finished" podID="acd5b194-6f4c-4cef-9342-95ec643504a8" containerID="16a2e71f1d15ce3ff9f6f018373fb47fa120f2a5f6efc009c5bcc7fc7cc91655" exitCode=0
Jul 02 23:49:52.743551 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.743391    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"
Jul 02 23:49:52.743551 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.743427    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx" event={"ID":"acd5b194-6f4c-4cef-9342-95ec643504a8","Type":"ContainerDied","Data":"16a2e71f1d15ce3ff9f6f018373fb47fa120f2a5f6efc009c5bcc7fc7cc91655"}
Jul 02 23:49:52.743551 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.743485    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx" event={"ID":"acd5b194-6f4c-4cef-9342-95ec643504a8","Type":"ContainerDied","Data":"47278af6092da07ba782fdcac566f0896f7b7b63414cf97408ac2795aae7fba8"}
Jul 02 23:49:52.743551 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.743510    2580 scope.go:117] "RemoveContainer" containerID="16a2e71f1d15ce3ff9f6f018373fb47fa120f2a5f6efc009c5bcc7fc7cc91655"
Jul 02 23:49:52.753057 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.753041    2580 scope.go:117] "RemoveContainer" containerID="d980b02f96a70bb25e74a3830e6ed9a246b71fdc651f761d5daff7fb2dbcbafd"
Jul 02 23:49:52.761396 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.761380    2580 scope.go:117] "RemoveContainer" containerID="16a2e71f1d15ce3ff9f6f018373fb47fa120f2a5f6efc009c5bcc7fc7cc91655"
Jul 02 23:49:52.761667 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:49:52.761649    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16a2e71f1d15ce3ff9f6f018373fb47fa120f2a5f6efc009c5bcc7fc7cc91655\": container with ID starting with 16a2e71f1d15ce3ff9f6f018373fb47fa120f2a5f6efc009c5bcc7fc7cc91655 not found: ID does not exist" containerID="16a2e71f1d15ce3ff9f6f018373fb47fa120f2a5f6efc009c5bcc7fc7cc91655"
Jul 02 23:49:52.761747 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.761675    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16a2e71f1d15ce3ff9f6f018373fb47fa120f2a5f6efc009c5bcc7fc7cc91655"} err="failed to get container status \"16a2e71f1d15ce3ff9f6f018373fb47fa120f2a5f6efc009c5bcc7fc7cc91655\": rpc error: code = NotFound desc = could not find container \"16a2e71f1d15ce3ff9f6f018373fb47fa120f2a5f6efc009c5bcc7fc7cc91655\": container with ID starting with 16a2e71f1d15ce3ff9f6f018373fb47fa120f2a5f6efc009c5bcc7fc7cc91655 not found: ID does not exist"
Jul 02 23:49:52.761747 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.761691    2580 scope.go:117] "RemoveContainer" containerID="d980b02f96a70bb25e74a3830e6ed9a246b71fdc651f761d5daff7fb2dbcbafd"
Jul 02 23:49:52.761964 ip-10-0-140-52 kubenswrapper[2580]: E0702 23:49:52.761944    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d980b02f96a70bb25e74a3830e6ed9a246b71fdc651f761d5daff7fb2dbcbafd\": container with ID starting with d980b02f96a70bb25e74a3830e6ed9a246b71fdc651f761d5daff7fb2dbcbafd not found: ID does not exist" containerID="d980b02f96a70bb25e74a3830e6ed9a246b71fdc651f761d5daff7fb2dbcbafd"
Jul 02 23:49:52.762002 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.761971    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d980b02f96a70bb25e74a3830e6ed9a246b71fdc651f761d5daff7fb2dbcbafd"} err="failed to get container status \"d980b02f96a70bb25e74a3830e6ed9a246b71fdc651f761d5daff7fb2dbcbafd\": rpc error: code = NotFound desc = could not find container \"d980b02f96a70bb25e74a3830e6ed9a246b71fdc651f761d5daff7fb2dbcbafd\": container with ID starting with d980b02f96a70bb25e74a3830e6ed9a246b71fdc651f761d5daff7fb2dbcbafd not found: ID does not exist"
Jul 02 23:49:52.765297 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.765275    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"]
Jul 02 23:49:52.769441 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:52.769423    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-fcd282-openshift-default-786946d94c-qqnfx"]
Jul 02 23:49:53.095600 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:53.095523    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qbcpb_006fb957-9603-4c51-b3b4-dc012737ec89/manager/0.log"
Jul 02 23:49:53.283481 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:53.283449    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acd5b194-6f4c-4cef-9342-95ec643504a8" path="/var/lib/kubelet/pods/acd5b194-6f4c-4cef-9342-95ec643504a8/volumes"
Jul 02 23:49:53.343064 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:53.343039    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d98bc9d58-svd4b_cfff93ba-acef-4836-be23-4663650be31e/copy-manifests/0.log"
Jul 02 23:49:53.349741 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:53.349673    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d98bc9d58-svd4b_cfff93ba-acef-4836-be23-4663650be31e/manager/0.log"
Jul 02 23:49:53.579371 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:53.579342    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-qj4zs_7b374e4f-26ff-4f69-9ed0-a5f607a28270/postgres/0.log"
Jul 02 23:49:54.345906 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:54.345875    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g_0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb/pull/0.log"
Jul 02 23:49:54.353766 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:54.353742    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g_0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb/extract/0.log"
Jul 02 23:49:54.359417 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:54.359402    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g_0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb/util/0.log"
Jul 02 23:49:54.467509 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:54.467486    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn_2bbc37b4-d84e-44b5-a3f3-86cb542915b0/util/0.log"
Jul 02 23:49:54.473430 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:54.473408    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn_2bbc37b4-d84e-44b5-a3f3-86cb542915b0/pull/0.log"
Jul 02 23:49:54.479497 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:54.479479    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn_2bbc37b4-d84e-44b5-a3f3-86cb542915b0/extract/0.log"
Jul 02 23:49:54.587888 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:54.587854    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb_c0b9cf75-f964-4467-ace5-5362020b4be7/util/0.log"
Jul 02 23:49:54.594286 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:54.594263    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb_c0b9cf75-f964-4467-ace5-5362020b4be7/pull/0.log"
Jul 02 23:49:54.600367 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:54.600310    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb_c0b9cf75-f964-4467-ace5-5362020b4be7/extract/0.log"
Jul 02 23:49:54.707086 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:54.707060    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc_91527594-f1c6-4f49-bbd0-f862aa1d3477/util/0.log"
Jul 02 23:49:54.712748 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:54.712731    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc_91527594-f1c6-4f49-bbd0-f862aa1d3477/pull/0.log"
Jul 02 23:49:54.717948 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:54.717930    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc_91527594-f1c6-4f49-bbd0-f862aa1d3477/extract/0.log"
Jul 02 23:49:54.956679 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:54.956647    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-vwf7m_8aa3ce58-a92d-4963-bb7f-d81c64a010d8/manager/0.log"
Jul 02 23:49:55.073916 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:55.073888    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-cj4rk_292cfb62-98cd-426c-aab9-9d28ec65db6d/manager/0.log"
Jul 02 23:49:55.186969 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:55.186941    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-wqjwd_9850397a-ee60-40de-9e65-6b3ced3e0194/kuadrant-console-plugin/0.log"
Jul 02 23:49:55.308859 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:55.308782    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-zj7mb_ada81472-d26c-4ef8-b31d-4ebdf4dd9132/registry-server/0.log"
Jul 02 23:49:55.570290 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:55.570212    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-5czdx_8311e635-8c00-435c-af36-7b6a62cbba5d/limitador/0.log"
Jul 02 23:49:55.687036 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:55.687008    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-wsg8r_cc0ea9df-b88b-4bf6-9f29-d02f6a53f198/manager/0.log"
Jul 02 23:49:56.270297 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:56.270269    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b76b84765-5x656_a0fa7374-6c60-44ba-8775-e5d1fc779b0d/kube-auth-proxy/0.log"
Jul 02 23:49:56.502577 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:56.502545    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-c9ld8_23482fd4-bf4d-4bf7-a5ae-ab019df7f807/istio-proxy/1.log"
Jul 02 23:49:56.742665 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:56.742635    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-rbl9d_acc52d0a-7366-468b-8a0a-8b1240a7facd/payload-processing/0.log"
Jul 02 23:49:57.429881 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:57.429848    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-wvdzz_a10b5aa4-5198-42de-bda1-8054dd5caefa/main/0.log"
Jul 02 23:49:57.654993 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:57.654962    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-7ttdh_1d9a71e6-6de3-4d94-b255-2d0f5c30ead5/main/0.log"
Jul 02 23:49:57.767920 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:49:57.767887    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-sxstp_144b0851-31ea-4c6d-b3cb-e7256b934511/main/0.log"
Jul 02 23:50:04.900237 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:04.900207    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vdlx9_e1d3f804-fd8f-43fa-81ab-82a96cc020e0/global-pull-secret-syncer/0.log"
Jul 02 23:50:05.053800 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:05.053775    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-thnfg_220907ed-e9d9-4b0c-9b89-7be9fdd4552e/konnectivity-agent/0.log"
Jul 02 23:50:05.123412 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:05.123376    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-52.ec2.internal_5a709290dff23a904b036c05f5cc09e5/haproxy/0.log"
Jul 02 23:50:09.481271 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:09.481241    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g_0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb/extract/0.log"
Jul 02 23:50:09.507780 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:09.507755    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g_0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb/util/0.log"
Jul 02 23:50:09.529514 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:09.529487    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dr82g_0b8fc35a-4cf3-4c5d-b5b1-718a91be02eb/pull/0.log"
Jul 02 23:50:09.554442 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:09.554417    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn_2bbc37b4-d84e-44b5-a3f3-86cb542915b0/extract/0.log"
Jul 02 23:50:09.575135 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:09.575084    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn_2bbc37b4-d84e-44b5-a3f3-86cb542915b0/util/0.log"
Jul 02 23:50:09.597354 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:09.597331    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0kcvkn_2bbc37b4-d84e-44b5-a3f3-86cb542915b0/pull/0.log"
Jul 02 23:50:09.626226 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:09.626178    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb_c0b9cf75-f964-4467-ace5-5362020b4be7/extract/0.log"
Jul 02 23:50:09.645460 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:09.645434    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb_c0b9cf75-f964-4467-ace5-5362020b4be7/util/0.log"
Jul 02 23:50:09.667836 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:09.667814    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73525hb_c0b9cf75-f964-4467-ace5-5362020b4be7/pull/0.log"
Jul 02 23:50:09.695220 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:09.695192    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc_91527594-f1c6-4f49-bbd0-f862aa1d3477/extract/0.log"
Jul 02 23:50:09.715860 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:09.715834    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc_91527594-f1c6-4f49-bbd0-f862aa1d3477/util/0.log"
Jul 02 23:50:09.735938 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:09.735863    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vxqjc_91527594-f1c6-4f49-bbd0-f862aa1d3477/pull/0.log"
Jul 02 23:50:10.169182 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:10.169096    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-vwf7m_8aa3ce58-a92d-4963-bb7f-d81c64a010d8/manager/0.log"
Jul 02 23:50:10.194467 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:10.194441    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-cj4rk_292cfb62-98cd-426c-aab9-9d28ec65db6d/manager/0.log"
Jul 02 23:50:10.218742 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:10.218713    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-wqjwd_9850397a-ee60-40de-9e65-6b3ced3e0194/kuadrant-console-plugin/0.log"
Jul 02 23:50:10.254648 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:10.254602    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-zj7mb_ada81472-d26c-4ef8-b31d-4ebdf4dd9132/registry-server/0.log"
Jul 02 23:50:10.456722 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:10.456695    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-5czdx_8311e635-8c00-435c-af36-7b6a62cbba5d/limitador/0.log"
Jul 02 23:50:10.516758 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:10.516724    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-wsg8r_cc0ea9df-b88b-4bf6-9f29-d02f6a53f198/manager/0.log"
Jul 02 23:50:12.709832 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:12.709804    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n8wxc_a4881e10-2a01-489c-ae57-c0de17e56bd5/node-exporter/0.log"
Jul 02 23:50:12.745174 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:12.745144    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n8wxc_a4881e10-2a01-489c-ae57-c0de17e56bd5/kube-rbac-proxy/0.log"
Jul 02 23:50:12.773955 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:12.773933    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n8wxc_a4881e10-2a01-489c-ae57-c0de17e56bd5/init-textfile/0.log"
Jul 02 23:50:14.375168 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.375138    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"]
Jul 02 23:50:14.375570 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.375473    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a45c5458-50c5-4662-aaff-58837fe2c2e1" containerName="main"
Jul 02 23:50:14.375570 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.375484    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a45c5458-50c5-4662-aaff-58837fe2c2e1" containerName="main"
Jul 02 23:50:14.375570 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.375492    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="654acc03-ac32-484a-a540-f6d1807efee5" containerName="main"
Jul 02 23:50:14.375570 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.375498    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="654acc03-ac32-484a-a540-f6d1807efee5" containerName="main"
Jul 02 23:50:14.375570 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.375508    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acd5b194-6f4c-4cef-9342-95ec643504a8" containerName="istio-proxy"
Jul 02 23:50:14.375570 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.375514    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="acd5b194-6f4c-4cef-9342-95ec643504a8" containerName="istio-proxy"
Jul 02 23:50:14.375570 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.375526    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acd5b194-6f4c-4cef-9342-95ec643504a8" containerName="istio-proxy"
Jul 02 23:50:14.375570 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.375531    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="acd5b194-6f4c-4cef-9342-95ec643504a8" containerName="istio-proxy"
Jul 02 23:50:14.375860 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.375583    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="acd5b194-6f4c-4cef-9342-95ec643504a8" containerName="istio-proxy"
Jul 02 23:50:14.375860 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.375591    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="654acc03-ac32-484a-a540-f6d1807efee5" containerName="main"
Jul 02 23:50:14.375860 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.375598    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="acd5b194-6f4c-4cef-9342-95ec643504a8" containerName="istio-proxy"
Jul 02 23:50:14.375860 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.375604    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a45c5458-50c5-4662-aaff-58837fe2c2e1" containerName="main"
Jul 02 23:50:14.378628 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.378610    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.382766 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.382744    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vvcqv\"/\"openshift-service-ca.crt\""
Jul 02 23:50:14.384050 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.384031    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vvcqv\"/\"default-dockercfg-jlszs\""
Jul 02 23:50:14.384183 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.384053    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vvcqv\"/\"kube-root-ca.crt\""
Jul 02 23:50:14.392823 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.392800    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"]
Jul 02 23:50:14.546825 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.546784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b8f7aab7-5a1b-4717-896d-2bed3e152e88-proc\") pod \"perf-node-gather-daemonset-xmj72\" (UID: \"b8f7aab7-5a1b-4717-896d-2bed3e152e88\") " pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.546825 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.546830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8f7aab7-5a1b-4717-896d-2bed3e152e88-sys\") pod \"perf-node-gather-daemonset-xmj72\" (UID: \"b8f7aab7-5a1b-4717-896d-2bed3e152e88\") " pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.547075 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.546909    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8f7aab7-5a1b-4717-896d-2bed3e152e88-lib-modules\") pod \"perf-node-gather-daemonset-xmj72\" (UID: \"b8f7aab7-5a1b-4717-896d-2bed3e152e88\") " pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.547075 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.546931    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b8f7aab7-5a1b-4717-896d-2bed3e152e88-podres\") pod \"perf-node-gather-daemonset-xmj72\" (UID: \"b8f7aab7-5a1b-4717-896d-2bed3e152e88\") " pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.547075 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.546982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbzkr\" (UniqueName: \"kubernetes.io/projected/b8f7aab7-5a1b-4717-896d-2bed3e152e88-kube-api-access-jbzkr\") pod \"perf-node-gather-daemonset-xmj72\" (UID: \"b8f7aab7-5a1b-4717-896d-2bed3e152e88\") " pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.648006 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.647924    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b8f7aab7-5a1b-4717-896d-2bed3e152e88-proc\") pod \"perf-node-gather-daemonset-xmj72\" (UID: \"b8f7aab7-5a1b-4717-896d-2bed3e152e88\") " pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.648006 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.647969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8f7aab7-5a1b-4717-896d-2bed3e152e88-sys\") pod \"perf-node-gather-daemonset-xmj72\" (UID: \"b8f7aab7-5a1b-4717-896d-2bed3e152e88\") " pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.648360 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.648020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8f7aab7-5a1b-4717-896d-2bed3e152e88-lib-modules\") pod \"perf-node-gather-daemonset-xmj72\" (UID: \"b8f7aab7-5a1b-4717-896d-2bed3e152e88\") " pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.648360 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.648036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b8f7aab7-5a1b-4717-896d-2bed3e152e88-podres\") pod \"perf-node-gather-daemonset-xmj72\" (UID: \"b8f7aab7-5a1b-4717-896d-2bed3e152e88\") " pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.648360 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.648061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbzkr\" (UniqueName: \"kubernetes.io/projected/b8f7aab7-5a1b-4717-896d-2bed3e152e88-kube-api-access-jbzkr\") pod \"perf-node-gather-daemonset-xmj72\" (UID: \"b8f7aab7-5a1b-4717-896d-2bed3e152e88\") " pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.648360 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.648068    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b8f7aab7-5a1b-4717-896d-2bed3e152e88-proc\") pod \"perf-node-gather-daemonset-xmj72\" (UID: \"b8f7aab7-5a1b-4717-896d-2bed3e152e88\") " pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.648360 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.648152    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b8f7aab7-5a1b-4717-896d-2bed3e152e88-sys\") pod \"perf-node-gather-daemonset-xmj72\" (UID: \"b8f7aab7-5a1b-4717-896d-2bed3e152e88\") " pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.648360 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.648196    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b8f7aab7-5a1b-4717-896d-2bed3e152e88-lib-modules\") pod \"perf-node-gather-daemonset-xmj72\" (UID: \"b8f7aab7-5a1b-4717-896d-2bed3e152e88\") " pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.648360 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.648225    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b8f7aab7-5a1b-4717-896d-2bed3e152e88-podres\") pod \"perf-node-gather-daemonset-xmj72\" (UID: \"b8f7aab7-5a1b-4717-896d-2bed3e152e88\") " pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.657145 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.657097    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbzkr\" (UniqueName: \"kubernetes.io/projected/b8f7aab7-5a1b-4717-896d-2bed3e152e88-kube-api-access-jbzkr\") pod \"perf-node-gather-daemonset-xmj72\" (UID: \"b8f7aab7-5a1b-4717-896d-2bed3e152e88\") " pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.688253 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.688210    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:14.814837 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.814801    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"]
Jul 02 23:50:14.818146 ip-10-0-140-52 kubenswrapper[2580]: W0702 23:50:14.818096    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb8f7aab7_5a1b_4717_896d_2bed3e152e88.slice/crio-3c2d6c751597969dca5ded4dd6f08b28ae8970397c6144ebe15e13d29a3f7881 WatchSource:0}: Error finding container 3c2d6c751597969dca5ded4dd6f08b28ae8970397c6144ebe15e13d29a3f7881: Status 404 returned error can't find the container with id 3c2d6c751597969dca5ded4dd6f08b28ae8970397c6144ebe15e13d29a3f7881
Jul 02 23:50:14.857418 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:14.857382    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72" event={"ID":"b8f7aab7-5a1b-4717-896d-2bed3e152e88","Type":"ContainerStarted","Data":"3c2d6c751597969dca5ded4dd6f08b28ae8970397c6144ebe15e13d29a3f7881"}
Jul 02 23:50:15.864227 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:15.864192    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72" event={"ID":"b8f7aab7-5a1b-4717-896d-2bed3e152e88","Type":"ContainerStarted","Data":"d79e768373ec486a5de6f8af462a8cd924a3cbd69487000b4437b490089f2602"}
Jul 02 23:50:15.864623 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:15.864334    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:15.880613 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:15.880563    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72" podStartSLOduration=1.8805511799999999 podStartE2EDuration="1.88055118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 23:50:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 23:50:15.879893018 +0000 UTC m=+2361.133683018" watchObservedRunningTime="2026-07-02 23:50:15.88055118 +0000 UTC m=+2361.134341178"
Jul 02 23:50:17.334096 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:17.334067    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lhb24_d07d35c0-4de5-48b9-b2c2-890e8f479e17/dns/0.log"
Jul 02 23:50:17.355300 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:17.355242    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lhb24_d07d35c0-4de5-48b9-b2c2-890e8f479e17/kube-rbac-proxy/0.log"
Jul 02 23:50:17.437194 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:17.437169    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vhqb2_72a6430a-6d3b-489d-afb8-465243b1bf36/dns-node-resolver/0.log"
Jul 02 23:50:17.982602 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:17.982556    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6bf947ff87-xrbkj_c89ddf32-dea8-4da4-a6bc-738cafaa2bda/registry/0.log"
Jul 02 23:50:18.001625 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:18.001579    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-54qq8_b9e5b05d-952a-4a24-8221-9c64bfa92ec8/node-ca/0.log"
Jul 02 23:50:19.183764 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:19.183737    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b76b84765-5x656_a0fa7374-6c60-44ba-8775-e5d1fc779b0d/kube-auth-proxy/0.log"
Jul 02 23:50:19.236503 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:19.236470    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-c9ld8_23482fd4-bf4d-4bf7-a5ae-ab019df7f807/istio-proxy/0.log"
Jul 02 23:50:19.263780 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:19.263734    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-c9ld8_23482fd4-bf4d-4bf7-a5ae-ab019df7f807/istio-proxy/1.log"
Jul 02 23:50:19.343743 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:19.343717    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-rbl9d_acc52d0a-7366-468b-8a0a-8b1240a7facd/payload-processing/0.log"
Jul 02 23:50:20.029038 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:20.029008    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5j8vq_10eb02f1-db27-4000-87ee-07331d835fda/serve-healthcheck-canary/0.log"
Jul 02 23:50:20.644664 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:20.644634    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f7h7z_98bf3fef-ce15-4556-abac-89874cf50e09/kube-rbac-proxy/0.log"
Jul 02 23:50:20.666363 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:20.666333    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f7h7z_98bf3fef-ce15-4556-abac-89874cf50e09/exporter/0.log"
Jul 02 23:50:20.686653 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:20.686621    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f7h7z_98bf3fef-ce15-4556-abac-89874cf50e09/extractor/0.log"
Jul 02 23:50:21.878704 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:21.878673    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vvcqv/perf-node-gather-daemonset-xmj72"
Jul 02 23:50:22.889949 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:22.889920    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-w2pkr_a00b6be7-7525-4621-b309-07f433f73161/manager/0.log"
Jul 02 23:50:23.026438 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:23.026406    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5tvj9_3dbab3c8-c572-4831-b03e-1a94243d20b3/manager/0.log"
Jul 02 23:50:23.351351 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:23.351321    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qbcpb_006fb957-9603-4c51-b3b4-dc012737ec89/manager/0.log"
Jul 02 23:50:23.507963 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:23.507930    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d98bc9d58-svd4b_cfff93ba-acef-4836-be23-4663650be31e/manager/0.log"
Jul 02 23:50:23.530957 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:23.530929    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d98bc9d58-svd4b_cfff93ba-acef-4836-be23-4663650be31e/copy-manifests/0.log"
Jul 02 23:50:23.612872 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:23.612792    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-qj4zs_7b374e4f-26ff-4f69-9ed0-a5f607a28270/postgres/0.log"
Jul 02 23:50:25.065415 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:25.065379    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b76b84765-5x656_a0fa7374-6c60-44ba-8775-e5d1fc779b0d/kube-auth-proxy/0.log"
Jul 02 23:50:25.099849 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:25.099817    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-c9ld8_23482fd4-bf4d-4bf7-a5ae-ab019df7f807/istio-proxy/0.log"
Jul 02 23:50:25.126344 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:25.126318    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-c9ld8_23482fd4-bf4d-4bf7-a5ae-ab019df7f807/istio-proxy/1.log"
Jul 02 23:50:25.190892 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:25.190857    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-rbl9d_acc52d0a-7366-468b-8a0a-8b1240a7facd/payload-processing/0.log"
Jul 02 23:50:26.575480 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:26.575446    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b76b84765-5x656_a0fa7374-6c60-44ba-8775-e5d1fc779b0d/kube-auth-proxy/0.log"
Jul 02 23:50:26.608674 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:26.608637    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-c9ld8_23482fd4-bf4d-4bf7-a5ae-ab019df7f807/istio-proxy/0.log"
Jul 02 23:50:26.641095 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:26.641064    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-c9ld8_23482fd4-bf4d-4bf7-a5ae-ab019df7f807/istio-proxy/1.log"
Jul 02 23:50:26.705474 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:26.705439    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-rbl9d_acc52d0a-7366-468b-8a0a-8b1240a7facd/payload-processing/0.log"
Jul 02 23:50:29.852321 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:29.852294    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-5jwtj_e4e0cee9-b9cf-4a41-9e60-6592f430b731/migrator/0.log"
Jul 02 23:50:29.871222 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:29.871195    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-5jwtj_e4e0cee9-b9cf-4a41-9e60-6592f430b731/graceful-termination/0.log"
Jul 02 23:50:31.463652 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:31.463623    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5jsxq_eed7310e-b623-4eca-91a1-c4c6bc3df056/kube-multus-additional-cni-plugins/0.log"
Jul 02 23:50:31.483977 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:31.483951    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5jsxq_eed7310e-b623-4eca-91a1-c4c6bc3df056/egress-router-binary-copy/0.log"
Jul 02 23:50:31.506482 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:31.506454    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5jsxq_eed7310e-b623-4eca-91a1-c4c6bc3df056/cni-plugins/0.log"
Jul 02 23:50:31.527563 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:31.527538    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5jsxq_eed7310e-b623-4eca-91a1-c4c6bc3df056/bond-cni-plugin/0.log"
Jul 02 23:50:31.546923 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:31.546894    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5jsxq_eed7310e-b623-4eca-91a1-c4c6bc3df056/routeoverride-cni/0.log"
Jul 02 23:50:31.566890 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:31.566861    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5jsxq_eed7310e-b623-4eca-91a1-c4c6bc3df056/whereabouts-cni-bincopy/0.log"
Jul 02 23:50:31.586563 ip-10-0-140-52 kubenswrapper[2580]: I0702 23:50:31.586532    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5jsxq_eed7310e-b623-4eca-91a1-c4c6bc3df056/whereabouts-cni/0.log"