Jul 09 15:00:19.577492 ip-10-0-129-152 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 09 15:00:19.577504 ip-10-0-129-152 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 09 15:00:19.577513 ip-10-0-129-152 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 09 15:00:19.577853 ip-10-0-129-152 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 09 15:00:29.729112 ip-10-0-129-152 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 09 15:00:29.729133 ip-10-0-129-152 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 3ec9059d3ea34983820c0a7f620faf97 --
Jul 09 15:02:28.479117 ip-10-0-129-152 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 15:02:28.905675 ip-10-0-129-152 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 15:02:28.905675 ip-10-0-129-152 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 15:02:28.905675 ip-10-0-129-152 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 15:02:28.905675 ip-10-0-129-152 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 15:02:28.905675 ip-10-0-129-152 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 15:02:28.905675 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.891792    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 15:02:28.905675 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896716    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 15:02:28.905675 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896731    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 15:02:28.905675 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896737    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 15:02:28.905675 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896742    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 15:02:28.905675 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896746    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 15:02:28.905675 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896749    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 15:02:28.905675 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896753    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 15:02:28.905675 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896756    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896758    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896761    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896763    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896766    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896769    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896771    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896773    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896776    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896778    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896781    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896784    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896788    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896792    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896795    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896798    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896801    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896804    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896808    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 15:02:28.906932 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896812    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896815    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896818    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896820    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896824    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896826    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896830    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896832    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896835    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896838    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896842    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896845    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896848    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896850    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896853    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896855    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896858    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896860    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896863    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896866    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 15:02:28.907797 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896869    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896872    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896874    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896877    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896879    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896882    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896884    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896887    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896890    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896893    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896895    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896898    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896900    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896903    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896905    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896907    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896910    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896913    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896915    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896918    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 15:02:28.908636 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896922    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896925    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896927    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896930    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896933    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896936    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896939    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896942    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896945    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896947    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896950    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896953    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896955    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896958    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896960    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896963    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896966    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896968    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896970    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896973    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 15:02:28.910592 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896975    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896978    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896980    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896983    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896985    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896988    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896990    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896992    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896995    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896997    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.896999    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897002    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897005    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897716    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897723    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897725    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897731    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897735    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897739    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 15:02:28.911156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897742    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897745    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897747    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897750    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897753    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897757    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897761    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897763    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897766    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897768    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897771    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897774    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897776    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897779    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897781    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897784    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897787    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897790    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897792    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897794    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 15:02:29.021194 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897797    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 15:02:28.965725 ip-10-0-129-152 systemd[1]: Started Kubernetes Kubelet.
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897800    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897803    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897805    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897808    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897811    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897813    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897816    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897818    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897821    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897824    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897826    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897829    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897832    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897836    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897838    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897842    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897845    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897848    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 15:02:29.022313 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897850    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897854    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897856    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897859    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897862    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897864    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897867    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897870    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897878    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897880    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897884    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897886    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897889    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897891    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897894    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897897    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897899    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897901    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897904    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 15:02:29.023368 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897906    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897909    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897911    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897914    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897916    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897919    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897922    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897924    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897927    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897930    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897932    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897934    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897937    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897939    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897942    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897946    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897948    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897951    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897953    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 15:02:29.024441 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897956    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897958    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897962    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897964    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897967    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897969    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897972    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897974    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897977    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897979    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897982    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897985    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897987    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897990    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897992    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.897994    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898355    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898360    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898363    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898367    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 15:02:29.025286 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898371    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898374    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898377    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898380    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898383    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898386    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898389    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898392    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898395    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898397    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898418    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898421    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898424    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898427    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898432    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898434    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898438    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898441    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898443    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898446    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 15:02:29.025962 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898449    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898451    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898454    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898457    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898460    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898463    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898465    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898469    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898471    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898474    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898477    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898480    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898482    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898485    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898487    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898490    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898493    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898495    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898500    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 15:02:29.026786 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898502    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898505    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898507    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898510    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898512    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898514    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898517    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898521    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898523    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898526    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898528    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898531    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898533    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898536    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898538    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898541    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898543    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898546    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898548    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898551    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 15:02:29.027544 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898553    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898556    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898558    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898561    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898563    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898566    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898569    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898571    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898575    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898578    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898581    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898586    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898589    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898593    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898596    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898598    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898601    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898604    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898607    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 15:02:29.028089 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898610    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898613    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898616    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898618    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898621    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898623    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898625    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898628    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898630    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898633    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898635    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898638    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898640    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898643    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898645    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898647    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.898650    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898723    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898737    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898747    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898753    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 15:02:29.028691 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898757    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898761    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898764    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898770    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898773    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898776    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898780    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898783    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898786    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898789    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898792    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898796    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898799    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898802    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898805    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898808    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898811    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898814    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898817    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898821    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898824    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898828    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898831    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898834    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 15:02:29.029315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898837    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898839    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898843    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898845    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898849    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898852    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898855    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898858    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898861    2579 flags.go:64] FLAG: --enable-server="true"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898864    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898869    2579 flags.go:64] FLAG: --event-burst="100"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898873    2579 flags.go:64] FLAG: --event-qps="50"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898876    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898879    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898882    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898885    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898888    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898891    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898894    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898898    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898901    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898904    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898907    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898910    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898913    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 15:02:29.030349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898916    2579 flags.go:64] FLAG: --feature-gates=""
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898920    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898923    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898926    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898929    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898931    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898934    2579 flags.go:64] FLAG: --help="false"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898937    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-129-152.ec2.internal"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898940    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898943    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898946    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898949    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898952    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898955    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898958    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898961    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898964    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898966    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898971    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898974    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898977    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898980    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898983    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898985    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 15:02:29.031659 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898988    2579 flags.go:64] FLAG: --lock-file=""
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898991    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898995    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.898998    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899003    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899006    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899009    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899012    2579 flags.go:64] FLAG: --logging-format="text"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899015    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899018    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899021    2579 flags.go:64] FLAG: --manifest-url=""
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899023    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899028    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899031    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899035    2579 flags.go:64] FLAG: --max-pods="110"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899038    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899040    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899043    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899046    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899049    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899052    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899055    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899062    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899065    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899068    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 15:02:29.032725 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899071    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899075    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899081    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899083    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899086    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899089    2579 flags.go:64] FLAG: --port="10250"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899092    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899095    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ab9cc26b6dbfedd1"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899098    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899103    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899105    2579 flags.go:64] FLAG: --register-node="true"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899108    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899112    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899114    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899117    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899120    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899123    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899126    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899129    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899132    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899135    2579 flags.go:64] FLAG: --runonce="false"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899138    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899140    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899143    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899147    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 15:02:29.033800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899149    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899152    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899155    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899158    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899161    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899164    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899167    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899170    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899173    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899177    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899180    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899185    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899188    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899190    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899194    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899197    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899201    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899204    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899207    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899209    2579 flags.go:64] FLAG: --v="2"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899213    2579 flags.go:64] FLAG: --version="false"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899217    2579 flags.go:64] FLAG: --vmodule=""
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899221    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.899224    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899334    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 15:02:29.034858 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899338    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899341    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899343    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899346    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899348    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899351    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899353    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899356    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899358    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899361    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899363    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899366    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899368    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899371    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899373    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899376    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899380    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899383    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899385    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 15:02:29.035934 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899388    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899390    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899392    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899395    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899399    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899414    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899417    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899420    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899422    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899426    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899430    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899432    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899435    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899438    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899440    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899443    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899445    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899448    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899450    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899453    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 15:02:29.036785 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899455    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899458    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899460    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899463    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899466    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899468    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899471    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899473    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899476    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899480    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899483    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899486    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899488    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899491    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899493    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899495    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899499    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899501    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899504    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899507    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 15:02:29.037646 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899509    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899512    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899514    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899517    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899519    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899522    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899524    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899527    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899530    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899532    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899536    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899540    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899543    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899546    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899549    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899551    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899555    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899558    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899561    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 15:02:29.038494 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899564    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899566    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899571    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899575    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899578    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899581    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899584    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899586    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899589    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899594    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899597    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899600    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899603    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899605    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899608    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899610    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899612    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899615    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899618    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 15:02:29.042717 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.899620    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 15:02:29.043566 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.901124    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 15:02:29.043566 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.910222    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 09 15:02:29.043566 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.910240    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 15:02:29.043566 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.910260    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 15:02:29.043566 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.910266    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 15:02:29.043566 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910593    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 15:02:29.043566 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910605    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 15:02:29.043566 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910611    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 15:02:29.043566 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910616    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 15:02:29.043566 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910620    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 15:02:29.043566 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910624    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 15:02:29.043566 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910629    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 15:02:29.043566 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910632    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 15:02:29.043566 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910636    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 15:02:29.043566 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910641    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910645    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910649    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910653    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910656    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910660    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910664    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910669    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910672    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910676    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910680    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910684    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910688    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910692    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910696    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910700    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910703    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910707    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910711    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910715    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 15:02:29.197542 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910723    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910727    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910732    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910736    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910740    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910745    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910749    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910753    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910757    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910761    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910764    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910768    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910773    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910777    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910781    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910786    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910789    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910793    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910796    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910798    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 15:02:29.216834 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910801    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910804    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910806    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910809    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910812    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910815    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910817    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910820    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910822    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910825    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910829    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910833    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910836    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910839    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910842    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910844    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910847    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910849    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910852    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 15:02:30.288210 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910855    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910858    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910860    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910863    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910865    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910868    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910870    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910873    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910875    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910878    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910880    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910883    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910886    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910888    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910891    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910893    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910896    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910898    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910901    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910903    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 15:02:30.289481 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910906    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 15:02:30.290343 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910908    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 15:02:30.290343 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910911    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 15:02:30.290343 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910913    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 15:02:30.290343 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910918    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 15:02:30.290343 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910923    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 15:02:30.290343 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910927    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 15:02:30.290343 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910930    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 15:02:30.290343 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910933    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 15:02:30.290343 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910936    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 15:02:30.290343 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.910939    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 15:02:30.290343 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.910944    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 15:02:30.290343 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911080    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 15:02:30.290343 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911084    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 15:02:30.290343 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911087    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911090    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911092    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911095    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911097    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911100    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911103    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911105    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911107    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911110    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911114    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911116    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911119    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911121    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911124    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911126    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911129    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911134    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911136    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911139    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 15:02:30.291220 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911142    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911144    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911148    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911150    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911153    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911155    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911158    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911161    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911164    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911166    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911169    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911172    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911174    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911177    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911180    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911182    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911185    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911187    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 15:02:30.292123 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911190    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911194    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911197    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911200    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911203    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911208    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911211    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911213    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911216    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911220    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911222    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911226    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911229    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911231    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911234    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911236    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911239    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911242    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911244    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 15:02:30.293019 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911247    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911249    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911252    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911254    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911257    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911259    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911263    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911266    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911269    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911271    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911274    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911276    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911279    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911282    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911284    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911286    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911289    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911291    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911294    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911298    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 15:02:30.293908 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911301    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911303    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911305    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911308    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911312    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911314    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911317    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911319    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911322    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911324    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911327    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911329    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911332    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911335    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911337    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911340    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911342    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911345    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911347    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 15:02:30.294782 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:28.911350    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 15:02:30.295653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.911355    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 15:02:30.295653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.911481    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 15:02:30.295653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.914211    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 15:02:30.295653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.915616    2579 server.go:1019] "Starting client certificate rotation"
Jul 09 15:02:30.295653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.915712    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 15:02:30.295653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.915749    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 15:02:30.295653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.918081    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 15:02:30.295653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.919206    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 15:02:30.295653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.932904    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 09 15:02:30.295653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.935529    2579 log.go:25] "Validated CRI v1 image API"
Jul 09 15:02:30.295653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.936197    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 15:02:30.295653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.939173    2579 fs.go:135] Filesystem UUIDs: map[34741652-48cf-4ab8-9ff8-bef5474f8f08:/dev/nvme0n1p4 5ca979e2-48a9-423d-8689-69ed26c536a2:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 09 15:02:30.296309 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.939188    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 15:02:30.296523 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.944684    2579 manager.go:216] Machine: {Timestamp:2026-07-09 15:02:28.943463513 +0000 UTC m=+0.347174770 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200630 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e6db600230c1f078c460bb29d7d24 SystemUUID:ec2e6db6-0023-0c1f-078c-460bb29d7d24 BootID:3ec9059d-3ea3-4983-820c-0a7f620faf97 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8f:49:49:17:89 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8f:49:49:17:89 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:e4:3a:99:81:7e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 15:02:30.296523 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.944781    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 15:02:30.296523 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.944859    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 15:02:30.296523 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.947316    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 15:02:30.296523 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.947339    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-152.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 15:02:30.296523 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.948159    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 15:02:30.296523 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.948171    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 15:02:30.296523 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.948178    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.948194    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.949542    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.952681    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.952785    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.953013    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.953029    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.953053    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.953066    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.953086    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.953484    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.953500    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.956769    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.958472    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.960352    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.960378    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.960557    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.960573    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.960580    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.960585    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.960593    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.960598    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.961107    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.961113    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.961121    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.961126    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.961134    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.961142    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.961791    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.961797    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:28.964454    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-152.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:28.964526    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.964835    2579 server.go:1268] "Started kubelet"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.964852    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-152.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.964911    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.964986    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.965067    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.966577    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.967861    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 15:02:30.297176 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:28.972800    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.973020    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.973033    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:28.972184    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-152.ec2.internal.18c0a740bf1d15bf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-152.ec2.internal,UID:ip-10-0-129-152.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-152.ec2.internal,},FirstTimestamp:2026-07-09 15:02:28.964816319 +0000 UTC m=+0.368527578,LastTimestamp:2026-07-09 15:02:28.964816319 +0000 UTC m=+0.368527578,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-152.ec2.internal,}"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:28.973129    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-152.ec2.internal\" not found"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.973638    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.973632    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.973684    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.973810    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.973818    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.974123    2579 factory.go:153] Registering CRI-O factory
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.974136    2579 factory.go:223] Registration of the crio container factory successfully
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.974197    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.974207    2579 factory.go:55] Registering systemd factory
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.974215    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.974237    2579 factory.go:103] Registering Raw factory
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.974249    2579 manager.go:1196] Started watching for new ooms in manager
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.974640    2579 manager.go:349] Starting recovery of all containers
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:28.978470    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:28.978737    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-152.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.983820    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lnc5v"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.985297    2579 manager.go:354] Recovery completed
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.991381    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.992698    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lnc5v"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.994003    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.994035    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.994049    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasSufficientPID"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.994596    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.994602    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.994605    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.994638    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.995214    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 15:02:30.299100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.995237    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.995246    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasSufficientPID"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.995906    2579 policy_none.go:49] "None policy: Start"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.995924    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.995934    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:28.996423    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-152.ec2.internal.18c0a740c0daad0b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-152.ec2.internal,UID:ip-10-0-129-152.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-152.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-152.ec2.internal,},FirstTimestamp:2026-07-09 15:02:28.994018571 +0000 UTC m=+0.397729828,LastTimestamp:2026-07-09 15:02:28.994018571 +0000 UTC m=+0.397729828,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-152.ec2.internal,}"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:28.997026    2579 policy_none.go:47] "Start"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.060960    2579 manager.go:337] "Starting Device Plugin manager"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:29.061119    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.061132    2579 server.go:87] "Starting device plugin registration server"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.061331    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.061341    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.061435    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.061514    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.061523    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:29.062044    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:29.062088    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-152.ec2.internal\" not found"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.153209    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.154341    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.154388    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.154443    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:29.154482    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.156745    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.165438    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.166233    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.166260    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.166269    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasSufficientPID"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.166292    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:29.175632    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:29.175650    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-152.ec2.internal\": node \"ip-10-0-129-152.ec2.internal\" not found"
Jul 09 15:02:30.300601 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:29.200877    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-152.ec2.internal\" not found"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.325604    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-152.ec2.internal"]
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.325674    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.326839    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.326867    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.326876    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasSufficientPID"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.328278    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.328376    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.328410    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.329055    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.329083    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.329099    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasSufficientPID"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.329101    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.329121    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.329130    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasSufficientPID"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.330450    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.330476    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.331132    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.331157    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.331169    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeHasSufficientPID"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.345807    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6e771d8b6b733fef68914549cc470703-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal\" (UID: \"6e771d8b6b733fef68914549cc470703\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.345830    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e771d8b6b733fef68914549cc470703-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal\" (UID: \"6e771d8b6b733fef68914549cc470703\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.345848    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e37ba62c1a936e070da56c82d756fe21-config\") pod \"kube-apiserver-proxy-ip-10-0-129-152.ec2.internal\" (UID: \"e37ba62c1a936e070da56c82d756fe21\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:30.351305    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-152.ec2.internal\" not found" node="ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:30.355630    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-152.ec2.internal\" not found" node="ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.428786 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:30.371200    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-152.ec2.internal\" not found"
Jul 09 15:02:30.525873 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.446651    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6e771d8b6b733fef68914549cc470703-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal\" (UID: \"6e771d8b6b733fef68914549cc470703\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.525873 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.446679    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e771d8b6b733fef68914549cc470703-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal\" (UID: \"6e771d8b6b733fef68914549cc470703\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.525873 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.446719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e37ba62c1a936e070da56c82d756fe21-config\") pod \"kube-apiserver-proxy-ip-10-0-129-152.ec2.internal\" (UID: \"e37ba62c1a936e070da56c82d756fe21\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.525873 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.446751    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6e771d8b6b733fef68914549cc470703-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal\" (UID: \"6e771d8b6b733fef68914549cc470703\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.525873 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.446768    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e37ba62c1a936e070da56c82d756fe21-config\") pod \"kube-apiserver-proxy-ip-10-0-129-152.ec2.internal\" (UID: \"e37ba62c1a936e070da56c82d756fe21\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.525873 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.446754    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e771d8b6b733fef68914549cc470703-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal\" (UID: \"6e771d8b6b733fef68914549cc470703\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.525873 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:30.471605    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-152.ec2.internal\" not found"
Jul 09 15:02:30.611267 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:30.571808    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-152.ec2.internal\" not found"
Jul 09 15:02:30.686076 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.653285    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.686076 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.658516    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-152.ec2.internal"
Jul 09 15:02:30.686076 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:30.672282    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-152.ec2.internal\" not found"
Jul 09 15:02:30.812821 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:30.772796    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-152.ec2.internal\" not found"
Jul 09 15:02:30.921389 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:30.873294    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-152.ec2.internal\" not found"
Jul 09 15:02:31.049560 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.944040    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 15:02:31.049560 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.985383    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 15:02:31.049560 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.985508    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 15:02:31.049560 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:30.985534    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 15:02:31.049560 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.025379    2579 apiserver.go:52] "Watching apiserver"
Jul 09 15:02:31.049560 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.044101    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal"
Jul 09 15:02:31.049560 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.044105    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 15:02:31.049560 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.045418    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 15:02:31.049560 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.046527    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-4pqph","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj","openshift-image-registry/node-ca-ncj7q","openshift-multus/multus-additional-cni-plugins-t8h8j","openshift-multus/multus-wbcx6","openshift-ovn-kubernetes/ovnkube-node-n8ctb","kube-system/global-pull-secret-syncer-6ltzh","kube-system/konnectivity-agent-r6r56","openshift-cluster-node-tuning-operator/tuned-f64g4","openshift-dns/node-resolver-jrt9w","openshift-multus/network-metrics-daemon-9swxx","openshift-network-diagnostics/network-check-target-qw9p7"]
Jul 09 15:02:31.049560 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.048824    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4pqph"
Jul 09 15:02:31.051756 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.049629    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.051756 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.050664    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ncj7q"
Jul 09 15:02:31.051756 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.051530    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rxzcb\""
Jul 09 15:02:31.051756 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.051643    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.054728 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.054468    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.055591 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.055177    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 15:02:31.055591 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.055206    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 15:02:31.055591 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.055355    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 15:02:31.055909 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.055672    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 15:02:31.056063 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.055918    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 15:02:31.056063 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.056031    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.056081    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.056232    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.056243    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.056297    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.056306    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gncsm\""
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.056646    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.056962    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-88kj7\""
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.057152    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.057157    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.057155    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.057281    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.057438    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-n2jr2\""
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.058372    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.058647    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.058733    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.059292    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zk69p\""
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.059924    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r6r56"
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.060998    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.062377    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jrt9w"
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.063436    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.063474    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.064160    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal"]
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.064212    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.064243    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.064783    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.064838    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-152.ec2.internal"
Jul 09 15:02:31.173382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.065254    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.066363    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 14:57:30 +0000 UTC" deadline="2028-02-22 17:08:18.536834497 +0000 UTC"
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.066386    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14234h5m47.470450745s"
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.073012    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.073531    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.076032    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.076061    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.076252    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.076299    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.076497    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.076511    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4fm9c\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.076544    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.076726    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.076743    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rkj5l\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.076865    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h2xsm\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.076984    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.077048    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.079411    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.081608    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b64nd\""
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.097818    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kcjr4"
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.109578    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-152.ec2.internal"]
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.110175    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.113266    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kcjr4"
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.144976    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151394    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6l8l\" (UniqueName: \"kubernetes.io/projected/1f898583-f94c-47ca-bddf-daed3be54740-kube-api-access-v6l8l\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151419    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4792ad3-2b68-4342-91e0-cc307d60e42a-cni-binary-copy\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.174524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151436    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-run-ovn-kubernetes\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.175976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151449    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-cni-bin\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.175976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151501    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-modprobe-d\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.175976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151527    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1f898583-f94c-47ca-bddf-daed3be54740-cni-binary-copy\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.175976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151546    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-run-k8s-cni-cncf-io\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.175976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151568    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1c59bea2-03b5-4242-8f9a-9510da7d4eca-ovnkube-config\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.175976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151582    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-sysconfig\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.175976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151596    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-sys\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.175976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151610    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-var-lib-kubelet\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.175976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151628    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ce9d4e7-f0cf-49b3-b925-12dc259bdd85-tmp-dir\") pod \"node-resolver-jrt9w\" (UID: \"7ce9d4e7-f0cf-49b3-b925-12dc259bdd85\") " pod="openshift-dns/node-resolver-jrt9w"
Jul 09 15:02:31.175976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151646    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zszz6\" (UniqueName: \"kubernetes.io/projected/d1e3c8c7-9fed-433d-879d-4763eb8459b5-kube-api-access-zszz6\") pod \"iptables-alerter-4pqph\" (UID: \"d1e3c8c7-9fed-433d-879d-4763eb8459b5\") " pod="openshift-network-operator/iptables-alerter-4pqph"
Jul 09 15:02:31.175976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151661    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4792ad3-2b68-4342-91e0-cc307d60e42a-system-cni-dir\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.175976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151679    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-kubernetes\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.175976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151728    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b5ln\" (UniqueName: \"kubernetes.io/projected/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-kube-api-access-4b5ln\") pod \"network-metrics-daemon-9swxx\" (UID: \"5fb7d2e1-0cc8-4827-a59c-59487768f9d3\") " pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:31.175976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151744    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d4792ad3-2b68-4342-91e0-cc307d60e42a-os-release\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.176506 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151763    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-slash\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.176506 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151776    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt9s8\" (UniqueName: \"kubernetes.io/projected/1c59bea2-03b5-4242-8f9a-9510da7d4eca-kube-api-access-tt9s8\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.176506 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151792    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-sysctl-conf\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.176506 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151812    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a973d8db-5cd4-490c-af42-c496f3ced07d-serviceca\") pod \"node-ca-ncj7q\" (UID: \"a973d8db-5cd4-490c-af42-c496f3ced07d\") " pod="openshift-image-registry/node-ca-ncj7q"
Jul 09 15:02:31.176506 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151845    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-var-lib-cni-multus\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.176506 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151860    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-hostroot\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.176506 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151879    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-systemd-units\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.176506 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151893    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d1e3c8c7-9fed-433d-879d-4763eb8459b5-iptables-alerter-script\") pod \"iptables-alerter-4pqph\" (UID: \"d1e3c8c7-9fed-433d-879d-4763eb8459b5\") " pod="openshift-network-operator/iptables-alerter-4pqph"
Jul 09 15:02:31.176506 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151907    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.176506 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151921    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-socket-dir\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.176506 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151935    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-device-dir\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.176506 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.151963    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-run-openvswitch\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.176506 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152000    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.177165 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152017    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2dg6\" (UniqueName: \"kubernetes.io/projected/a973d8db-5cd4-490c-af42-c496f3ced07d-kube-api-access-h2dg6\") pod \"node-ca-ncj7q\" (UID: \"a973d8db-5cd4-490c-af42-c496f3ced07d\") " pod="openshift-image-registry/node-ca-ncj7q"
Jul 09 15:02:31.177165 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152037    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d4792ad3-2b68-4342-91e0-cc307d60e42a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.177165 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152052    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-etc-openvswitch\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.177165 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152064    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1c59bea2-03b5-4242-8f9a-9510da7d4eca-env-overrides\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.177165 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152079    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4e083b0-d118-451b-8655-30df5353d5ed-kubelet-config\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:31.177165 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152096    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a973d8db-5cd4-490c-af42-c496f3ced07d-host\") pod \"node-ca-ncj7q\" (UID: \"a973d8db-5cd4-490c-af42-c496f3ced07d\") " pod="openshift-image-registry/node-ca-ncj7q"
Jul 09 15:02:31.177165 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1e3c8c7-9fed-433d-879d-4763eb8459b5-host-slash\") pod \"iptables-alerter-4pqph\" (UID: \"d1e3c8c7-9fed-433d-879d-4763eb8459b5\") " pod="openshift-network-operator/iptables-alerter-4pqph"
Jul 09 15:02:31.177165 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152138    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-run-ovn\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.177165 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-cni-netd\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.177165 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152169    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4e083b0-d118-451b-8655-30df5353d5ed-dbus\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:31.177165 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpmk7\" (UniqueName: \"kubernetes.io/projected/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-kube-api-access-lpmk7\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.177165 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152210    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1f898583-f94c-47ca-bddf-daed3be54740-multus-daemon-config\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.177165 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152225    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-registration-dir\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.177165 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152244    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:31.177770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152301    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-cnibin\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.177770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152316    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-os-release\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.177770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152333    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82r5h\" (UniqueName: \"kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h\") pod \"network-check-target-qw9p7\" (UID: \"68d94daa-3588-4e94-be77-28ab021314f5\") " pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:31.177770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152362    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.177770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152377    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4792ad3-2b68-4342-91e0-cc307d60e42a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.177770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152391    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-kubelet\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.177770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152405    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-multus-cni-dir\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.177770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152424    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7feb5924-8488-453b-8093-63183e5c9462-agent-certs\") pod \"konnectivity-agent-r6r56\" (UID: \"7feb5924-8488-453b-8093-63183e5c9462\") " pod="kube-system/konnectivity-agent-r6r56"
Jul 09 15:02:31.177770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152438    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ce9d4e7-f0cf-49b3-b925-12dc259bdd85-hosts-file\") pod \"node-resolver-jrt9w\" (UID: \"7ce9d4e7-f0cf-49b3-b925-12dc259bdd85\") " pod="openshift-dns/node-resolver-jrt9w"
Jul 09 15:02:31.177770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152452    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-multus-conf-dir\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.177770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152466    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-run-multus-certs\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.177770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152480    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7feb5924-8488-453b-8093-63183e5c9462-konnectivity-ca\") pod \"konnectivity-agent-r6r56\" (UID: \"7feb5924-8488-453b-8093-63183e5c9462\") " pod="kube-system/konnectivity-agent-r6r56"
Jul 09 15:02:31.177770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152500    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-sys-fs\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.177770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152515    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-log-socket\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.178433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152528    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1c59bea2-03b5-4242-8f9a-9510da7d4eca-ovnkube-script-lib\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.178433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152549    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-run\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.178433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152566    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-etc-kubernetes\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.178433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152585    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs\") pod \"network-metrics-daemon-9swxx\" (UID: \"5fb7d2e1-0cc8-4827-a59c-59487768f9d3\") " pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:31.178433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152602    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-tuned\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.178433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152618    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-run-systemd\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.178433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152634    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp799\" (UniqueName: \"kubernetes.io/projected/1e46b888-5ce8-43e0-858c-5545b28e9d9c-kube-api-access-dp799\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.178433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152660    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-sysctl-d\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.178433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152675    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-systemd\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.178433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152710    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-tmp\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.178433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152727    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qd62\" (UniqueName: \"kubernetes.io/projected/7ce9d4e7-f0cf-49b3-b925-12dc259bdd85-kube-api-access-8qd62\") pod \"node-resolver-jrt9w\" (UID: \"7ce9d4e7-f0cf-49b3-b925-12dc259bdd85\") " pod="openshift-dns/node-resolver-jrt9w"
Jul 09 15:02:31.178433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152741    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-multus-socket-dir-parent\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.178433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152755    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-host\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.178433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152771    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-run-netns\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.178998 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152792    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-var-lib-openvswitch\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.178998 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152894    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d4792ad3-2b68-4342-91e0-cc307d60e42a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.178998 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152913    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7z2s\" (UniqueName: \"kubernetes.io/projected/d4792ad3-2b68-4342-91e0-cc307d60e42a-kube-api-access-g7z2s\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.178998 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152930    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-system-cni-dir\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.178998 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152952    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-var-lib-cni-bin\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.178998 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152969    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-var-lib-kubelet\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.178998 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152984    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c59bea2-03b5-4242-8f9a-9510da7d4eca-ovn-node-metrics-cert\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.178998 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.152998    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-lib-modules\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.178998 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.153012    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4792ad3-2b68-4342-91e0-cc307d60e42a-cnibin\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.178998 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.153025    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-node-log\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.178998 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.153048    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-run-netns\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.296054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.193475    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 15:02:31.296054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253571    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1c59bea2-03b5-4242-8f9a-9510da7d4eca-env-overrides\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.296054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253596    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4e083b0-d118-451b-8655-30df5353d5ed-kubelet-config\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:31.296054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253613    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a973d8db-5cd4-490c-af42-c496f3ced07d-host\") pod \"node-ca-ncj7q\" (UID: \"a973d8db-5cd4-490c-af42-c496f3ced07d\") " pod="openshift-image-registry/node-ca-ncj7q"
Jul 09 15:02:31.296054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1e3c8c7-9fed-433d-879d-4763eb8459b5-host-slash\") pod \"iptables-alerter-4pqph\" (UID: \"d1e3c8c7-9fed-433d-879d-4763eb8459b5\") " pod="openshift-network-operator/iptables-alerter-4pqph"
Jul 09 15:02:31.296054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253675    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1e3c8c7-9fed-433d-879d-4763eb8459b5-host-slash\") pod \"iptables-alerter-4pqph\" (UID: \"d1e3c8c7-9fed-433d-879d-4763eb8459b5\") " pod="openshift-network-operator/iptables-alerter-4pqph"
Jul 09 15:02:31.296054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253705    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a973d8db-5cd4-490c-af42-c496f3ced07d-host\") pod \"node-ca-ncj7q\" (UID: \"a973d8db-5cd4-490c-af42-c496f3ced07d\") " pod="openshift-image-registry/node-ca-ncj7q"
Jul 09 15:02:31.296054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253722    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-run-ovn\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.296054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253708    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4e083b0-d118-451b-8655-30df5353d5ed-kubelet-config\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:31.296054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253746    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-cni-netd\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.296054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253764    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4e083b0-d118-451b-8655-30df5353d5ed-dbus\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:31.296054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253766    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-run-ovn\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.296054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253789    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpmk7\" (UniqueName: \"kubernetes.io/projected/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-kube-api-access-lpmk7\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.296054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253796    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-cni-netd\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.296054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253818    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1f898583-f94c-47ca-bddf-daed3be54740-multus-daemon-config\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.298266 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253842    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-registration-dir\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.298266 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253872    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:31.298266 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-cnibin\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.298266 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-os-release\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.298266 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253935    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82r5h\" (UniqueName: \"kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h\") pod \"network-check-target-qw9p7\" (UID: \"68d94daa-3588-4e94-be77-28ab021314f5\") " pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:31.298266 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253957    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.298266 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253979    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4792ad3-2b68-4342-91e0-cc307d60e42a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.298266 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254004    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-kubelet\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.298266 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254020    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-registration-dir\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.298266 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254027    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-multus-cni-dir\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.298266 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.254060    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 15:02:31.298266 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254066    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-cnibin\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.298266 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254066    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.298266 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253976    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4e083b0-d118-451b-8655-30df5353d5ed-dbus\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:31.298266 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254084    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-multus-cni-dir\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.298824 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.254122    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret podName:a4e083b0-d118-451b-8655-30df5353d5ed nodeName:}" failed. No retries permitted until 2026-07-09 15:02:31.754103544 +0000 UTC m=+2.087533506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret") pod "global-pull-secret-syncer-6ltzh" (UID: "a4e083b0-d118-451b-8655-30df5353d5ed") : object "kube-system"/"original-pull-secret" not registered
Jul 09 15:02:31.298824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254131    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-os-release\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.298824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254132    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-kubelet\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.298824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.253979    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1c59bea2-03b5-4242-8f9a-9510da7d4eca-env-overrides\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.298824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7feb5924-8488-453b-8093-63183e5c9462-agent-certs\") pod \"konnectivity-agent-r6r56\" (UID: \"7feb5924-8488-453b-8093-63183e5c9462\") " pod="kube-system/konnectivity-agent-r6r56"
Jul 09 15:02:31.298824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254173    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1f898583-f94c-47ca-bddf-daed3be54740-multus-daemon-config\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.298824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254181    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4792ad3-2b68-4342-91e0-cc307d60e42a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.298824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254180    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ce9d4e7-f0cf-49b3-b925-12dc259bdd85-hosts-file\") pod \"node-resolver-jrt9w\" (UID: \"7ce9d4e7-f0cf-49b3-b925-12dc259bdd85\") " pod="openshift-dns/node-resolver-jrt9w"
Jul 09 15:02:31.298824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254205    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-multus-conf-dir\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.298824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254220    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-run-multus-certs\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.298824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254226    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ce9d4e7-f0cf-49b3-b925-12dc259bdd85-hosts-file\") pod \"node-resolver-jrt9w\" (UID: \"7ce9d4e7-f0cf-49b3-b925-12dc259bdd85\") " pod="openshift-dns/node-resolver-jrt9w"
Jul 09 15:02:31.298824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254235    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7feb5924-8488-453b-8093-63183e5c9462-konnectivity-ca\") pod \"konnectivity-agent-r6r56\" (UID: \"7feb5924-8488-453b-8093-63183e5c9462\") " pod="kube-system/konnectivity-agent-r6r56"
Jul 09 15:02:31.298824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-sys-fs\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.298824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254269    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-multus-conf-dir\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.299504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254279    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-log-socket\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.299504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1c59bea2-03b5-4242-8f9a-9510da7d4eca-ovnkube-script-lib\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.299504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254306    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-run-multus-certs\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.299504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254313    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-run\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.299504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254332    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-etc-kubernetes\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.299504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254341    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-run\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.299504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254348    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs\") pod \"network-metrics-daemon-9swxx\" (UID: \"5fb7d2e1-0cc8-4827-a59c-59487768f9d3\") " pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:31.299504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254352    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-log-socket\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.299504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-tuned\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.299504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254385    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-run-systemd\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.299504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254400    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp799\" (UniqueName: \"kubernetes.io/projected/1e46b888-5ce8-43e0-858c-5545b28e9d9c-kube-api-access-dp799\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.299504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254414    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-sysctl-d\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.299504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254447    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-systemd\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.299504 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.254454    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:02:31.299504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-tmp\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.300254 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254475    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qd62\" (UniqueName: \"kubernetes.io/projected/7ce9d4e7-f0cf-49b3-b925-12dc259bdd85-kube-api-access-8qd62\") pod \"node-resolver-jrt9w\" (UID: \"7ce9d4e7-f0cf-49b3-b925-12dc259bdd85\") " pod="openshift-dns/node-resolver-jrt9w"
Jul 09 15:02:31.300254 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254400    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-sys-fs\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.300254 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.254491    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs podName:5fb7d2e1-0cc8-4827-a59c-59487768f9d3 nodeName:}" failed. No retries permitted until 2026-07-09 15:02:31.754478697 +0000 UTC m=+2.087908639 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs") pod "network-metrics-daemon-9swxx" (UID: "5fb7d2e1-0cc8-4827-a59c-59487768f9d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:02:31.300254 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254530    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 15:02:31.300254 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254576    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-multus-socket-dir-parent\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.300254 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254599    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-sysctl-d\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.300254 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254658    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-systemd\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.300254 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254658    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-etc-kubernetes\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.300254 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254535    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-multus-socket-dir-parent\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.300254 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-host\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.300254 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254737    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1c59bea2-03b5-4242-8f9a-9510da7d4eca-ovnkube-script-lib\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.300254 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254761    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-run-netns\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.300254 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254776    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-host\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.300254 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-var-lib-openvswitch\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.301041 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d4792ad3-2b68-4342-91e0-cc307d60e42a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.301041 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254809    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-run-netns\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.301041 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254815    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7feb5924-8488-453b-8093-63183e5c9462-konnectivity-ca\") pod \"konnectivity-agent-r6r56\" (UID: \"7feb5924-8488-453b-8093-63183e5c9462\") " pod="kube-system/konnectivity-agent-r6r56"
Jul 09 15:02:31.301041 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7z2s\" (UniqueName: \"kubernetes.io/projected/d4792ad3-2b68-4342-91e0-cc307d60e42a-kube-api-access-g7z2s\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.301041 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254846    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-var-lib-openvswitch\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.301041 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-system-cni-dir\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.301041 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254886    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-var-lib-cni-bin\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.301041 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254907    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-var-lib-kubelet\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.301041 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254925    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c59bea2-03b5-4242-8f9a-9510da7d4eca-ovn-node-metrics-cert\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.301041 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-lib-modules\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.301041 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254962    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4792ad3-2b68-4342-91e0-cc307d60e42a-cnibin\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.301041 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.254983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-node-log\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.301041 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255003    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-run-netns\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.301041 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6l8l\" (UniqueName: \"kubernetes.io/projected/1f898583-f94c-47ca-bddf-daed3be54740-kube-api-access-v6l8l\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.301612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255044    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-system-cni-dir\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.301612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255050    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4792ad3-2b68-4342-91e0-cc307d60e42a-cni-binary-copy\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.301612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255071    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-run-ovn-kubernetes\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.301612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255089    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-cni-bin\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.301612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255105    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-modprobe-d\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.301612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255121    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1f898583-f94c-47ca-bddf-daed3be54740-cni-binary-copy\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.301612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255136    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-run-k8s-cni-cncf-io\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.301612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1c59bea2-03b5-4242-8f9a-9510da7d4eca-ovnkube-config\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.301612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255168    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-sysconfig\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.301612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255177    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-lib-modules\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.301612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255184    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-sys\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.301612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255199    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-var-lib-kubelet\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.301612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255213    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ce9d4e7-f0cf-49b3-b925-12dc259bdd85-tmp-dir\") pod \"node-resolver-jrt9w\" (UID: \"7ce9d4e7-f0cf-49b3-b925-12dc259bdd85\") " pod="openshift-dns/node-resolver-jrt9w"
Jul 09 15:02:31.301612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255216    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4792ad3-2b68-4342-91e0-cc307d60e42a-cnibin\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.301612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zszz6\" (UniqueName: \"kubernetes.io/projected/d1e3c8c7-9fed-433d-879d-4763eb8459b5-kube-api-access-zszz6\") pod \"iptables-alerter-4pqph\" (UID: \"d1e3c8c7-9fed-433d-879d-4763eb8459b5\") " pod="openshift-network-operator/iptables-alerter-4pqph"
Jul 09 15:02:31.302742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255252    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4792ad3-2b68-4342-91e0-cc307d60e42a-system-cni-dir\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.302742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255252    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-node-log\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.302742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255254    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d4792ad3-2b68-4342-91e0-cc307d60e42a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.302742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255267    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-kubernetes\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.302742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255299    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-var-lib-cni-bin\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.302742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255327    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-var-lib-kubelet\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.302742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255347    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-run-ovn-kubernetes\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.302742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255368    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-cni-bin\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.302742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255436    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b5ln\" (UniqueName: \"kubernetes.io/projected/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-kube-api-access-4b5ln\") pod \"network-metrics-daemon-9swxx\" (UID: \"5fb7d2e1-0cc8-4827-a59c-59487768f9d3\") " pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:31.302742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255462    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d4792ad3-2b68-4342-91e0-cc307d60e42a-os-release\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.302742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255483    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-slash\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.302742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255503    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt9s8\" (UniqueName: \"kubernetes.io/projected/1c59bea2-03b5-4242-8f9a-9510da7d4eca-kube-api-access-tt9s8\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.302742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255523    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-sysctl-conf\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.302742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255541    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a973d8db-5cd4-490c-af42-c496f3ced07d-serviceca\") pod \"node-ca-ncj7q\" (UID: \"a973d8db-5cd4-490c-af42-c496f3ced07d\") " pod="openshift-image-registry/node-ca-ncj7q"
Jul 09 15:02:31.303539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255563    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-var-lib-cni-multus\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.303539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255588    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-hostroot\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.303539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255608    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-systemd-units\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.303539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255636    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d1e3c8c7-9fed-433d-879d-4763eb8459b5-iptables-alerter-script\") pod \"iptables-alerter-4pqph\" (UID: \"d1e3c8c7-9fed-433d-879d-4763eb8459b5\") " pod="openshift-network-operator/iptables-alerter-4pqph"
Jul 09 15:02:31.303539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.303539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255705    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-socket-dir\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.303539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255734    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-device-dir\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.303539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255759    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-run-openvswitch\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.303539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255792    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.303539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255817    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2dg6\" (UniqueName: \"kubernetes.io/projected/a973d8db-5cd4-490c-af42-c496f3ced07d-kube-api-access-h2dg6\") pod \"node-ca-ncj7q\" (UID: \"a973d8db-5cd4-490c-af42-c496f3ced07d\") " pod="openshift-image-registry/node-ca-ncj7q"
Jul 09 15:02:31.303539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255840    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d4792ad3-2b68-4342-91e0-cc307d60e42a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.303539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255871    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-etc-openvswitch\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.303539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255948    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-etc-openvswitch\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.303539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.255982    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-run-netns\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.304491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.256274    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4792ad3-2b68-4342-91e0-cc307d60e42a-cni-binary-copy\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.304491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.256348    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d4792ad3-2b68-4342-91e0-cc307d60e42a-os-release\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.304491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.256379    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-run-systemd\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.304491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.256396    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-slash\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.304491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.256632    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-sysctl-conf\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.304491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.256699    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-var-lib-cni-multus\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.304491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.256741    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-hostroot\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.304491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.256774    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-systemd-units\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.304491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.256808    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-run-openvswitch\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.304491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.256852    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.304491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.256828    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d1e3c8c7-9fed-433d-879d-4763eb8459b5-iptables-alerter-script\") pod \"iptables-alerter-4pqph\" (UID: \"d1e3c8c7-9fed-433d-879d-4763eb8459b5\") " pod="openshift-network-operator/iptables-alerter-4pqph"
Jul 09 15:02:31.304491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.256925    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-device-dir\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.304491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.256943    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e46b888-5ce8-43e0-858c-5545b28e9d9c-socket-dir\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.304491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.257091    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a973d8db-5cd4-490c-af42-c496f3ced07d-serviceca\") pod \"node-ca-ncj7q\" (UID: \"a973d8db-5cd4-490c-af42-c496f3ced07d\") " pod="openshift-image-registry/node-ca-ncj7q"
Jul 09 15:02:31.305083 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.257136    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c59bea2-03b5-4242-8f9a-9510da7d4eca-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.305083 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.257195    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-sys\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.305083 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.257254    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-modprobe-d\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.305083 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.257368    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1f898583-f94c-47ca-bddf-daed3be54740-host-run-k8s-cni-cncf-io\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.305083 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.257655    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1f898583-f94c-47ca-bddf-daed3be54740-cni-binary-copy\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.305083 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.257735    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-var-lib-kubelet\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.305083 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.257825    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4792ad3-2b68-4342-91e0-cc307d60e42a-system-cni-dir\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.305083 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.257855    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1c59bea2-03b5-4242-8f9a-9510da7d4eca-ovnkube-config\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.305083 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.257895    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-kubernetes\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.305083 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.257956    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-sysconfig\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.305083 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.258045    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ce9d4e7-f0cf-49b3-b925-12dc259bdd85-tmp-dir\") pod \"node-resolver-jrt9w\" (UID: \"7ce9d4e7-f0cf-49b3-b925-12dc259bdd85\") " pod="openshift-dns/node-resolver-jrt9w"
Jul 09 15:02:31.305083 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.258144    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-etc-tuned\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.305083 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.258305    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1c59bea2-03b5-4242-8f9a-9510da7d4eca-ovn-node-metrics-cert\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.305083 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.258384    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d4792ad3-2b68-4342-91e0-cc307d60e42a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.305083 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.258392    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-tmp\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.429816 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.258438    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7feb5924-8488-453b-8093-63183e5c9462-agent-certs\") pod \"konnectivity-agent-r6r56\" (UID: \"7feb5924-8488-453b-8093-63183e5c9462\") " pod="kube-system/konnectivity-agent-r6r56"
Jul 09 15:02:31.429816 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.263830    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 15:02:31.429816 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.267048    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpmk7\" (UniqueName: \"kubernetes.io/projected/9fb37dc4-8bd0-4e08-b9ce-f92002526d2d-kube-api-access-lpmk7\") pod \"tuned-f64g4\" (UID: \"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d\") " pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.429816 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.273422    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 15:02:31.429816 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.273441    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 15:02:31.429816 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.273454    2579 projected.go:196] Error preparing data for projected volume kube-api-access-82r5h for pod openshift-network-diagnostics/network-check-target-qw9p7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:02:31.429816 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.273519    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h podName:68d94daa-3588-4e94-be77-28ab021314f5 nodeName:}" failed. No retries permitted until 2026-07-09 15:02:31.773503128 +0000 UTC m=+2.106933093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-82r5h" (UniqueName: "kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h") pod "network-check-target-qw9p7" (UID: "68d94daa-3588-4e94-be77-28ab021314f5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:02:31.429816 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.273633    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r6r56"
Jul 09 15:02:31.429816 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.274632    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2dg6\" (UniqueName: \"kubernetes.io/projected/a973d8db-5cd4-490c-af42-c496f3ced07d-kube-api-access-h2dg6\") pod \"node-ca-ncj7q\" (UID: \"a973d8db-5cd4-490c-af42-c496f3ced07d\") " pod="openshift-image-registry/node-ca-ncj7q"
Jul 09 15:02:31.429816 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.277165    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt9s8\" (UniqueName: \"kubernetes.io/projected/1c59bea2-03b5-4242-8f9a-9510da7d4eca-kube-api-access-tt9s8\") pod \"ovnkube-node-n8ctb\" (UID: \"1c59bea2-03b5-4242-8f9a-9510da7d4eca\") " pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.429816 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.278342    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qd62\" (UniqueName: \"kubernetes.io/projected/7ce9d4e7-f0cf-49b3-b925-12dc259bdd85-kube-api-access-8qd62\") pod \"node-resolver-jrt9w\" (UID: \"7ce9d4e7-f0cf-49b3-b925-12dc259bdd85\") " pod="openshift-dns/node-resolver-jrt9w"
Jul 09 15:02:31.429816 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.278412    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zszz6\" (UniqueName: \"kubernetes.io/projected/d1e3c8c7-9fed-433d-879d-4763eb8459b5-kube-api-access-zszz6\") pod \"iptables-alerter-4pqph\" (UID: \"d1e3c8c7-9fed-433d-879d-4763eb8459b5\") " pod="openshift-network-operator/iptables-alerter-4pqph"
Jul 09 15:02:31.429816 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.278458    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6l8l\" (UniqueName: \"kubernetes.io/projected/1f898583-f94c-47ca-bddf-daed3be54740-kube-api-access-v6l8l\") pod \"multus-wbcx6\" (UID: \"1f898583-f94c-47ca-bddf-daed3be54740\") " pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.429816 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.279097    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b5ln\" (UniqueName: \"kubernetes.io/projected/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-kube-api-access-4b5ln\") pod \"network-metrics-daemon-9swxx\" (UID: \"5fb7d2e1-0cc8-4827-a59c-59487768f9d3\") " pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:31.429816 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.279288    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp799\" (UniqueName: \"kubernetes.io/projected/1e46b888-5ce8-43e0-858c-5545b28e9d9c-kube-api-access-dp799\") pod \"aws-ebs-csi-driver-node-pwppj\" (UID: \"1e46b888-5ce8-43e0-858c-5545b28e9d9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.431597 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.280041    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7z2s\" (UniqueName: \"kubernetes.io/projected/d4792ad3-2b68-4342-91e0-cc307d60e42a-kube-api-access-g7z2s\") pod \"multus-additional-cni-plugins-t8h8j\" (UID: \"d4792ad3-2b68-4342-91e0-cc307d60e42a\") " pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.431597 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.301587    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f64g4"
Jul 09 15:02:31.431597 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.307739    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jrt9w"
Jul 09 15:02:31.431597 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:31.323128    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e771d8b6b733fef68914549cc470703.slice/crio-85f775ed5e481f8519e45d24455c3fcf2f8208d0615c30a81f83531341f17928 WatchSource:0}: Error finding container 85f775ed5e481f8519e45d24455c3fcf2f8208d0615c30a81f83531341f17928: Status 404 returned error can't find the container with id 85f775ed5e481f8519e45d24455c3fcf2f8208d0615c30a81f83531341f17928
Jul 09 15:02:31.431597 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:31.323666    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode37ba62c1a936e070da56c82d756fe21.slice/crio-cea1edc9b51801190612b6595990aa7cf488ad8d353e77c2ed70f79f45b41032 WatchSource:0}: Error finding container cea1edc9b51801190612b6595990aa7cf488ad8d353e77c2ed70f79f45b41032: Status 404 returned error can't find the container with id cea1edc9b51801190612b6595990aa7cf488ad8d353e77c2ed70f79f45b41032
Jul 09 15:02:31.431597 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:31.328833    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb37dc4_8bd0_4e08_b9ce_f92002526d2d.slice/crio-f2320d09acf4c6aae3f2841ec74c428c8a7ac847a04795b76f8d5370d1fa89b2 WatchSource:0}: Error finding container f2320d09acf4c6aae3f2841ec74c428c8a7ac847a04795b76f8d5370d1fa89b2: Status 404 returned error can't find the container with id f2320d09acf4c6aae3f2841ec74c428c8a7ac847a04795b76f8d5370d1fa89b2
Jul 09 15:02:31.431597 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.332344    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 15:02:31.431597 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.398350    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4pqph"
Jul 09 15:02:31.431597 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.405379    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj"
Jul 09 15:02:31.524664 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.466492    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ncj7q"
Jul 09 15:02:31.524664 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:31.475308    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda973d8db_5cd4_490c_af42_c496f3ced07d.slice/crio-7a46b4425e8a0357c0fa21d0f0d8afd556cefb081d9a4d3c942f895f5a9b2378 WatchSource:0}: Error finding container 7a46b4425e8a0357c0fa21d0f0d8afd556cefb081d9a4d3c942f895f5a9b2378: Status 404 returned error can't find the container with id 7a46b4425e8a0357c0fa21d0f0d8afd556cefb081d9a4d3c942f895f5a9b2378
Jul 09 15:02:31.524664 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.495894    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t8h8j"
Jul 09 15:02:31.524664 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:31.505276    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4792ad3_2b68_4342_91e0_cc307d60e42a.slice/crio-649b523b73136ed8e69db91b978f5f7db0b80584c3f775578f0040ed8b2a5663 WatchSource:0}: Error finding container 649b523b73136ed8e69db91b978f5f7db0b80584c3f775578f0040ed8b2a5663: Status 404 returned error can't find the container with id 649b523b73136ed8e69db91b978f5f7db0b80584c3f775578f0040ed8b2a5663
Jul 09 15:02:31.628533 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.525501    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wbcx6"
Jul 09 15:02:31.628533 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:31.533796    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f898583_f94c_47ca_bddf_daed3be54740.slice/crio-918d0c5d6ea9371e81d093c44f90b5f0f426ab8637d45b97e8ae9b4eb82fe29c WatchSource:0}: Error finding container 918d0c5d6ea9371e81d093c44f90b5f0f426ab8637d45b97e8ae9b4eb82fe29c: Status 404 returned error can't find the container with id 918d0c5d6ea9371e81d093c44f90b5f0f426ab8637d45b97e8ae9b4eb82fe29c
Jul 09 15:02:31.628533 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.540140    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:31.628533 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:31.548910    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c59bea2_03b5_4242_8f9a_9510da7d4eca.slice/crio-4ff3208ccf6d2f59eaf3dfe95cddd8bb3b7314e702ab58c4409de7a627949557 WatchSource:0}: Error finding container 4ff3208ccf6d2f59eaf3dfe95cddd8bb3b7314e702ab58c4409de7a627949557: Status 404 returned error can't find the container with id 4ff3208ccf6d2f59eaf3dfe95cddd8bb3b7314e702ab58c4409de7a627949557
Jul 09 15:02:31.735409 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:02:31.735381    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7feb5924_8488_453b_8093_63183e5c9462.slice/crio-4aed2dc769326ff57ce2f7ee6dc47ad8146b0c80443acc16116146f3887b6000 WatchSource:0}: Error finding container 4aed2dc769326ff57ce2f7ee6dc47ad8146b0c80443acc16116146f3887b6000: Status 404 returned error can't find the container with id 4aed2dc769326ff57ce2f7ee6dc47ad8146b0c80443acc16116146f3887b6000
Jul 09 15:02:31.761989 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.761959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:31.762141 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.762007    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs\") pod \"network-metrics-daemon-9swxx\" (UID: \"5fb7d2e1-0cc8-4827-a59c-59487768f9d3\") " pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:31.762141 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.762130    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:02:31.762255 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.762190    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs podName:5fb7d2e1-0cc8-4827-a59c-59487768f9d3 nodeName:}" failed. No retries permitted until 2026-07-09 15:02:32.76217075 +0000 UTC m=+3.095600693 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs") pod "network-metrics-daemon-9swxx" (UID: "5fb7d2e1-0cc8-4827-a59c-59487768f9d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:02:31.762255 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.762133    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 15:02:31.762379 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.762279    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret podName:a4e083b0-d118-451b-8655-30df5353d5ed nodeName:}" failed. No retries permitted until 2026-07-09 15:02:32.762250276 +0000 UTC m=+3.095680232 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret") pod "global-pull-secret-syncer-6ltzh" (UID: "a4e083b0-d118-451b-8655-30df5353d5ed") : object "kube-system"/"original-pull-secret" not registered
Jul 09 15:02:31.862988 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:31.862910    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82r5h\" (UniqueName: \"kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h\") pod \"network-check-target-qw9p7\" (UID: \"68d94daa-3588-4e94-be77-28ab021314f5\") " pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:31.863160 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.863097    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 15:02:31.863160 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.863124    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 15:02:31.863160 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.863138    2579 projected.go:196] Error preparing data for projected volume kube-api-access-82r5h for pod openshift-network-diagnostics/network-check-target-qw9p7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:02:31.863270 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:31.863206    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h podName:68d94daa-3588-4e94-be77-28ab021314f5 nodeName:}" failed. No retries permitted until 2026-07-09 15:02:32.86318494 +0000 UTC m=+3.196614903 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-82r5h" (UniqueName: "kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h") pod "network-check-target-qw9p7" (UID: "68d94daa-3588-4e94-be77-28ab021314f5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:02:32.037760 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.037664    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 15:02:32.116009 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.115893    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 14:57:31 +0000 UTC" deadline="2027-12-12 07:55:17.202413141 +0000 UTC"
Jul 09 15:02:32.116009 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.115955    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12496h52m45.086462344s"
Jul 09 15:02:32.228854 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.228215    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:32.228854 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:32.228352    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:02:32.238695 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.238624    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r6r56" event={"ID":"7feb5924-8488-453b-8093-63183e5c9462","Type":"ContainerStarted","Data":"4aed2dc769326ff57ce2f7ee6dc47ad8146b0c80443acc16116146f3887b6000"}
Jul 09 15:02:32.244291 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.244211    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wbcx6" event={"ID":"1f898583-f94c-47ca-bddf-daed3be54740","Type":"ContainerStarted","Data":"918d0c5d6ea9371e81d093c44f90b5f0f426ab8637d45b97e8ae9b4eb82fe29c"}
Jul 09 15:02:32.246472 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.246443    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ncj7q" event={"ID":"a973d8db-5cd4-490c-af42-c496f3ced07d","Type":"ContainerStarted","Data":"7a46b4425e8a0357c0fa21d0f0d8afd556cefb081d9a4d3c942f895f5a9b2378"}
Jul 09 15:02:32.249552 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.249478    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj" event={"ID":"1e46b888-5ce8-43e0-858c-5545b28e9d9c","Type":"ContainerStarted","Data":"f7491df2f0276d1fcdb2004b24144f9fa245449f0580fc820c2b788336959191"}
Jul 09 15:02:32.252829 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.252763    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4pqph" event={"ID":"d1e3c8c7-9fed-433d-879d-4763eb8459b5","Type":"ContainerStarted","Data":"9c1abef97d52eac9d7023efdac28f0dd994473deda1758b2e7d6adcde546b2d8"}
Jul 09 15:02:32.256173 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.256152    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jrt9w" event={"ID":"7ce9d4e7-f0cf-49b3-b925-12dc259bdd85","Type":"ContainerStarted","Data":"64018704d8f2fe4bbc72fe7285bb5e9ec41233af422e3ee8c18e8409787782ad"}
Jul 09 15:02:32.262057 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.261962    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb" event={"ID":"1c59bea2-03b5-4242-8f9a-9510da7d4eca","Type":"ContainerStarted","Data":"4ff3208ccf6d2f59eaf3dfe95cddd8bb3b7314e702ab58c4409de7a627949557"}
Jul 09 15:02:32.267781 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.267752    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8h8j" event={"ID":"d4792ad3-2b68-4342-91e0-cc307d60e42a","Type":"ContainerStarted","Data":"649b523b73136ed8e69db91b978f5f7db0b80584c3f775578f0040ed8b2a5663"}
Jul 09 15:02:32.275750 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.275720    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-152.ec2.internal" event={"ID":"e37ba62c1a936e070da56c82d756fe21","Type":"ContainerStarted","Data":"cea1edc9b51801190612b6595990aa7cf488ad8d353e77c2ed70f79f45b41032"}
Jul 09 15:02:32.282902 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.282872    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal" event={"ID":"6e771d8b6b733fef68914549cc470703","Type":"ContainerStarted","Data":"85f775ed5e481f8519e45d24455c3fcf2f8208d0615c30a81f83531341f17928"}
Jul 09 15:02:32.295770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.294853    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f64g4" event={"ID":"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d","Type":"ContainerStarted","Data":"f2320d09acf4c6aae3f2841ec74c428c8a7ac847a04795b76f8d5370d1fa89b2"}
Jul 09 15:02:32.753180 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.753102    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 15:02:32.771924 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.771888    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:32.772087 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.771960    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs\") pod \"network-metrics-daemon-9swxx\" (UID: \"5fb7d2e1-0cc8-4827-a59c-59487768f9d3\") " pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:32.772146 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:32.772114    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:02:32.772217 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:32.772181    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs podName:5fb7d2e1-0cc8-4827-a59c-59487768f9d3 nodeName:}" failed. No retries permitted until 2026-07-09 15:02:34.772160874 +0000 UTC m=+5.105590818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs") pod "network-metrics-daemon-9swxx" (UID: "5fb7d2e1-0cc8-4827-a59c-59487768f9d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:02:32.772604 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:32.772574    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 15:02:32.772710 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:32.772630    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret podName:a4e083b0-d118-451b-8655-30df5353d5ed nodeName:}" failed. No retries permitted until 2026-07-09 15:02:34.772613945 +0000 UTC m=+5.106043890 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret") pod "global-pull-secret-syncer-6ltzh" (UID: "a4e083b0-d118-451b-8655-30df5353d5ed") : object "kube-system"/"original-pull-secret" not registered
Jul 09 15:02:32.872894 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:32.872859    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82r5h\" (UniqueName: \"kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h\") pod \"network-check-target-qw9p7\" (UID: \"68d94daa-3588-4e94-be77-28ab021314f5\") " pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:32.873078 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:32.873053    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 15:02:32.873078 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:32.873076    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 15:02:32.873187 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:32.873088    2579 projected.go:196] Error preparing data for projected volume kube-api-access-82r5h for pod openshift-network-diagnostics/network-check-target-qw9p7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:02:32.873187 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:32.873147    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h podName:68d94daa-3588-4e94-be77-28ab021314f5 nodeName:}" failed. No retries permitted until 2026-07-09 15:02:34.873129882 +0000 UTC m=+5.206559848 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-82r5h" (UniqueName: "kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h") pod "network-check-target-qw9p7" (UID: "68d94daa-3588-4e94-be77-28ab021314f5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:02:33.116248 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:33.116121    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 14:57:31 +0000 UTC" deadline="2027-12-13 23:40:10.835625163 +0000 UTC"
Jul 09 15:02:33.116248 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:33.116189    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12536h37m37.719440003s"
Jul 09 15:02:33.225439 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:33.225402    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:33.225624 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:33.225536    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:02:33.225866 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:33.225644    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:33.225866 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:33.225761    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:02:34.227896 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:34.227253    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:34.227896 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:34.227388    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:02:34.794445 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:34.793772    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs\") pod \"network-metrics-daemon-9swxx\" (UID: \"5fb7d2e1-0cc8-4827-a59c-59487768f9d3\") " pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:34.794445 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:34.793865    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:34.794445 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:34.794008    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 15:02:34.794445 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:34.794069    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret podName:a4e083b0-d118-451b-8655-30df5353d5ed nodeName:}" failed. No retries permitted until 2026-07-09 15:02:38.794052362 +0000 UTC m=+9.127482318 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret") pod "global-pull-secret-syncer-6ltzh" (UID: "a4e083b0-d118-451b-8655-30df5353d5ed") : object "kube-system"/"original-pull-secret" not registered
Jul 09 15:02:34.794445 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:34.794217    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:02:34.794445 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:34.794283    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs podName:5fb7d2e1-0cc8-4827-a59c-59487768f9d3 nodeName:}" failed. No retries permitted until 2026-07-09 15:02:38.794265138 +0000 UTC m=+9.127695087 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs") pod "network-metrics-daemon-9swxx" (UID: "5fb7d2e1-0cc8-4827-a59c-59487768f9d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:02:34.895349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:34.895216    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82r5h\" (UniqueName: \"kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h\") pod \"network-check-target-qw9p7\" (UID: \"68d94daa-3588-4e94-be77-28ab021314f5\") " pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:34.895545 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:34.895381    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 15:02:34.895545 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:34.895407    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 15:02:34.895545 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:34.895421    2579 projected.go:196] Error preparing data for projected volume kube-api-access-82r5h for pod openshift-network-diagnostics/network-check-target-qw9p7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:02:34.895545 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:34.895486    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h podName:68d94daa-3588-4e94-be77-28ab021314f5 nodeName:}" failed. No retries permitted until 2026-07-09 15:02:38.895467292 +0000 UTC m=+9.228897236 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-82r5h" (UniqueName: "kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h") pod "network-check-target-qw9p7" (UID: "68d94daa-3588-4e94-be77-28ab021314f5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:02:35.225299 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:35.225211    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:35.225460 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:35.225378    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:02:35.225520 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:35.225485    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:35.225623 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:35.225596    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:02:36.225139 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:36.225105    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:36.225604 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:36.225260    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:02:37.225766 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:37.225459    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:37.225766 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:37.225459    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:37.225766 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:37.225597    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:02:37.225766 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:37.225707    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:02:38.226024 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:38.225941    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:38.226465 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:38.226093    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:02:38.828088 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:38.828038    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:38.828271 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:38.828103    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs\") pod \"network-metrics-daemon-9swxx\" (UID: \"5fb7d2e1-0cc8-4827-a59c-59487768f9d3\") " pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:38.828271 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:38.828226    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:02:38.828377 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:38.828283    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs podName:5fb7d2e1-0cc8-4827-a59c-59487768f9d3 nodeName:}" failed. No retries permitted until 2026-07-09 15:02:46.828263281 +0000 UTC m=+17.161693223 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs") pod "network-metrics-daemon-9swxx" (UID: "5fb7d2e1-0cc8-4827-a59c-59487768f9d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:02:38.828609 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:38.828526    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 15:02:38.828609 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:38.828580    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret podName:a4e083b0-d118-451b-8655-30df5353d5ed nodeName:}" failed. No retries permitted until 2026-07-09 15:02:46.828564552 +0000 UTC m=+17.161994501 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret") pod "global-pull-secret-syncer-6ltzh" (UID: "a4e083b0-d118-451b-8655-30df5353d5ed") : object "kube-system"/"original-pull-secret" not registered
Jul 09 15:02:38.928586 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:38.928552    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82r5h\" (UniqueName: \"kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h\") pod \"network-check-target-qw9p7\" (UID: \"68d94daa-3588-4e94-be77-28ab021314f5\") " pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:38.928781 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:38.928744    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 15:02:38.928781 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:38.928772    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 15:02:38.928910 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:38.928785    2579 projected.go:196] Error preparing data for projected volume kube-api-access-82r5h for pod openshift-network-diagnostics/network-check-target-qw9p7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:02:38.928910 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:38.928848    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h podName:68d94daa-3588-4e94-be77-28ab021314f5 nodeName:}" failed. No retries permitted until 2026-07-09 15:02:46.928828701 +0000 UTC m=+17.262258666 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-82r5h" (UniqueName: "kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h") pod "network-check-target-qw9p7" (UID: "68d94daa-3588-4e94-be77-28ab021314f5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:02:39.225807 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:39.225707    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:39.225807 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:39.225733    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:39.226035 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:39.225855    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:02:39.226035 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:39.225977    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:02:40.227196 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:40.226912    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:40.227196 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:40.227051    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:02:41.226662 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:41.225706    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:41.226662 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:41.226169    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:02:41.226662 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:41.225706    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:41.226992 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:41.226730    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:02:42.225656 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:42.225621    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:42.226108 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:42.225785    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:02:43.225237 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:43.225196    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:43.225237 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:43.225224    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:43.225471 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:43.225325    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:02:43.225471 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:43.225397    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:02:44.224991 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:44.224962    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:44.225410 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:44.225110    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:02:45.225379 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:45.225337    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:45.225840 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:45.225337    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:45.225840 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:45.225462    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:02:45.225840 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:45.225515    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:02:46.225486 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:46.225402    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:46.225960 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:46.225557    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:02:46.889723 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:46.889667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs\") pod \"network-metrics-daemon-9swxx\" (UID: \"5fb7d2e1-0cc8-4827-a59c-59487768f9d3\") " pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:46.889948 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:46.889766    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:46.889948 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:46.889815    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:02:46.889948 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:46.889884    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 15:02:46.889948 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:46.889902    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs podName:5fb7d2e1-0cc8-4827-a59c-59487768f9d3 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:02.889879395 +0000 UTC m=+33.223309345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs") pod "network-metrics-daemon-9swxx" (UID: "5fb7d2e1-0cc8-4827-a59c-59487768f9d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:02:46.889948 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:46.889922    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret podName:a4e083b0-d118-451b-8655-30df5353d5ed nodeName:}" failed. No retries permitted until 2026-07-09 15:03:02.889911572 +0000 UTC m=+33.223341515 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret") pod "global-pull-secret-syncer-6ltzh" (UID: "a4e083b0-d118-451b-8655-30df5353d5ed") : object "kube-system"/"original-pull-secret" not registered
Jul 09 15:02:46.991102 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:46.991064    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82r5h\" (UniqueName: \"kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h\") pod \"network-check-target-qw9p7\" (UID: \"68d94daa-3588-4e94-be77-28ab021314f5\") " pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:46.991308 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:46.991267    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 15:02:46.991308 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:46.991292    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 15:02:46.991308 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:46.991305    2579 projected.go:196] Error preparing data for projected volume kube-api-access-82r5h for pod openshift-network-diagnostics/network-check-target-qw9p7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:02:46.991457 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:46.991372    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h podName:68d94daa-3588-4e94-be77-28ab021314f5 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:02.991353415 +0000 UTC m=+33.324783363 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-82r5h" (UniqueName: "kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h") pod "network-check-target-qw9p7" (UID: "68d94daa-3588-4e94-be77-28ab021314f5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:02:47.225536 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:47.225440    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:47.226035 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:47.225568    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:02:47.226035 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:47.225618    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:47.226035 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:47.225729    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:02:48.225286 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:48.225193    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:48.225483 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:48.225342    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:02:49.225401 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:49.225364    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:49.225827 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:49.225372    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:49.225827 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:49.225479    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:02:49.225827 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:49.225572    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:02:50.225604 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:50.225572    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:50.226067 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:50.225699    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:02:51.225966 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:51.225884    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:51.226289 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:51.225884    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:51.226289 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:51.225998    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:02:51.226289 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:51.226075    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:02:52.225505 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.225188    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:52.225657 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:52.225611    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:02:52.352829 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.352753    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:02:52.353513 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.353133    2579 generic.go:356] "Generic (PLEG): container finished" podID="1c59bea2-03b5-4242-8f9a-9510da7d4eca" containerID="b19c576a655fbba5db8ac5219ba7514d699f2feb47292c8a836945362abd1de0" exitCode=1
Jul 09 15:02:52.353513 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.353200    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb" event={"ID":"1c59bea2-03b5-4242-8f9a-9510da7d4eca","Type":"ContainerStarted","Data":"5691e46524d2d223a3fb154ce0f38f70c2986e9e38bd59612ef85d66603c7c1c"}
Jul 09 15:02:52.353513 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.353280    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb" event={"ID":"1c59bea2-03b5-4242-8f9a-9510da7d4eca","Type":"ContainerStarted","Data":"19a7d05c4e63602fdc591da3ea61ab6da94fbf47528a8af4c369cfd83b0f1a04"}
Jul 09 15:02:52.353513 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.353327    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb" event={"ID":"1c59bea2-03b5-4242-8f9a-9510da7d4eca","Type":"ContainerStarted","Data":"d0359b4bb67356c09bdf833485a2bce444b22a0597f3c43e91221b9f375fabd1"}
Jul 09 15:02:52.353513 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.353366    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb" event={"ID":"1c59bea2-03b5-4242-8f9a-9510da7d4eca","Type":"ContainerStarted","Data":"2accd2b8b18e277195151e4a517863a8ab66c0fe23e263761549fa495c4613e0"}
Jul 09 15:02:52.353513 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.353396    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb" event={"ID":"1c59bea2-03b5-4242-8f9a-9510da7d4eca","Type":"ContainerDied","Data":"b19c576a655fbba5db8ac5219ba7514d699f2feb47292c8a836945362abd1de0"}
Jul 09 15:02:52.353513 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.353446    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb" event={"ID":"1c59bea2-03b5-4242-8f9a-9510da7d4eca","Type":"ContainerStarted","Data":"ddc6ad3566536dd8126ddc857a6eb19be2f633d39a92bc273ada891e57ec4d46"}
Jul 09 15:02:52.354803 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.354779    2579 generic.go:356] "Generic (PLEG): container finished" podID="d4792ad3-2b68-4342-91e0-cc307d60e42a" containerID="03fcff2bc380fd1051d7f11b7e6adff40624e47b994ad6e1d1d83e0b882c6d6b" exitCode=0
Jul 09 15:02:52.354915 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.354822    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8h8j" event={"ID":"d4792ad3-2b68-4342-91e0-cc307d60e42a","Type":"ContainerDied","Data":"03fcff2bc380fd1051d7f11b7e6adff40624e47b994ad6e1d1d83e0b882c6d6b"}
Jul 09 15:02:52.356458 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.356433    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-152.ec2.internal" event={"ID":"e37ba62c1a936e070da56c82d756fe21","Type":"ContainerStarted","Data":"22ad64862fb7e7cbae55b45b7aac28adad033617bfd6f67bc5b998f875d1ad7c"}
Jul 09 15:02:52.358007 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.357984    2579 generic.go:356] "Generic (PLEG): container finished" podID="6e771d8b6b733fef68914549cc470703" containerID="18275cbc5bff7f197b3b2b46b19eb1ba4add4b71ffeabe3be0d71f2885171b0c" exitCode=0
Jul 09 15:02:52.358092 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.358038    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal" event={"ID":"6e771d8b6b733fef68914549cc470703","Type":"ContainerDied","Data":"18275cbc5bff7f197b3b2b46b19eb1ba4add4b71ffeabe3be0d71f2885171b0c"}
Jul 09 15:02:52.359777 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.359753    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f64g4" event={"ID":"9fb37dc4-8bd0-4e08-b9ce-f92002526d2d","Type":"ContainerStarted","Data":"b6b1a2b488121f25ce1af13f57123d3f886c708b4c894a237be9285fa61e0da3"}
Jul 09 15:02:52.364606 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.364575    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r6r56" event={"ID":"7feb5924-8488-453b-8093-63183e5c9462","Type":"ContainerStarted","Data":"50eddd4f8898a56a4d76a0b3762452130b0b78735bc06ed4d01753a709112353"}
Jul 09 15:02:52.366076 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.366031    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wbcx6" event={"ID":"1f898583-f94c-47ca-bddf-daed3be54740","Type":"ContainerStarted","Data":"b1ae33129f71d599f95abc9db8ef89e15944c4fa45a8dec0fe1bf670da3474e4"}
Jul 09 15:02:52.367464 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.367439    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ncj7q" event={"ID":"a973d8db-5cd4-490c-af42-c496f3ced07d","Type":"ContainerStarted","Data":"1e3b2d90d6ffadbd7b92f9ea524749eebcf715a3aca615469da4c6783c1519a0"}
Jul 09 15:02:52.368904 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.368882    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj" event={"ID":"1e46b888-5ce8-43e0-858c-5545b28e9d9c","Type":"ContainerStarted","Data":"9c4622c93118369fc62f854f537152a23a796192308508fe688834e7d75d3283"}
Jul 09 15:02:52.370312 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.370288    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jrt9w" event={"ID":"7ce9d4e7-f0cf-49b3-b925-12dc259bdd85","Type":"ContainerStarted","Data":"40b9907f6040eeb1016065ee27c6b32717c9be33553ec2b360da3e17e4441d05"}
Jul 09 15:02:52.398380 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.398301    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wbcx6" podStartSLOduration=2.5908242599999998 podStartE2EDuration="22.398280393s" totalImagesPullingTime="19.807456133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:52.397972322 +0000 UTC m=+22.731402289" watchObservedRunningTime="2026-07-09 15:02:52.398280393 +0000 UTC m=+22.731710359"
Jul 09 15:02:52.414402 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.414344    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-r6r56" podStartSLOduration=12.059157123 podStartE2EDuration="22.414325813s" totalImagesPullingTime="10.35516869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:52.413858855 +0000 UTC m=+22.747288821" watchObservedRunningTime="2026-07-09 15:02:52.414325813 +0000 UTC m=+22.747755777"
Jul 09 15:02:52.433159 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.433109    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jrt9w" podStartSLOduration=2.424598992 podStartE2EDuration="22.433092878s" totalImagesPullingTime="20.008493886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:52.432535792 +0000 UTC m=+22.765965758" watchObservedRunningTime="2026-07-09 15:02:52.433092878 +0000 UTC m=+22.766522842"
Jul 09 15:02:52.455779 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.455710    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-152.ec2.internal" podStartSLOduration=21.455673899 podStartE2EDuration="21.455673899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:52.454568245 +0000 UTC m=+22.787998210" watchObservedRunningTime="2026-07-09 15:02:52.455673899 +0000 UTC m=+22.789103867"
Jul 09 15:02:52.473328 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.473268    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ncj7q" podStartSLOduration=7.192176168 podStartE2EDuration="22.473253693s" totalImagesPullingTime="15.281077525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:52.472892312 +0000 UTC m=+22.806322278" watchObservedRunningTime="2026-07-09 15:02:52.473253693 +0000 UTC m=+22.806683698"
Jul 09 15:02:52.493631 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:52.493569    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-f64g4" podStartSLOduration=2.50538322 podStartE2EDuration="22.493547275s" totalImagesPullingTime="19.988164055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:52.493088557 +0000 UTC m=+22.826518526" watchObservedRunningTime="2026-07-09 15:02:52.493547275 +0000 UTC m=+22.826977240"
Jul 09 15:02:53.148011 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:53.147982    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 15:02:53.225867 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:53.225823    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:53.225867 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:53.225855    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:53.226091 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:53.225949    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:02:53.226091 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:53.226057    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:02:53.374537 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:53.374433    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj" event={"ID":"1e46b888-5ce8-43e0-858c-5545b28e9d9c","Type":"ContainerStarted","Data":"2c66e88bbe6d9da2994d936e63a074312314047b301ce7135badd8a1d6980124"}
Jul 09 15:02:53.375853 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:53.375824    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4pqph" event={"ID":"d1e3c8c7-9fed-433d-879d-4763eb8459b5","Type":"ContainerStarted","Data":"1585a1192403a6cb9accbd1ab108f65f8fe5e52cfae3684c5e21d7a8b6fab57a"}
Jul 09 15:02:53.377888 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:53.377852    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal" event={"ID":"6e771d8b6b733fef68914549cc470703","Type":"ContainerStarted","Data":"c7a97a4d8242cf990a41426ea69cc9e80e1195783f985d2e144cc35209db1ed0"}
Jul 09 15:02:53.411414 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:53.411359    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4pqph" podStartSLOduration=3.845903234 podStartE2EDuration="23.411341114s" totalImagesPullingTime="19.56543788s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:53.393909932 +0000 UTC m=+23.727339897" watchObservedRunningTime="2026-07-09 15:02:53.411341114 +0000 UTC m=+23.744771187"
Jul 09 15:02:53.411797 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:53.411772    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-152.ec2.internal" podStartSLOduration=22.411764707 podStartE2EDuration="22.411764707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:53.410973345 +0000 UTC m=+23.744403314" watchObservedRunningTime="2026-07-09 15:02:53.411764707 +0000 UTC m=+23.745194671"
Jul 09 15:02:54.147101 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:54.147000    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T15:02:53.148002084Z","UUID":"aa1bfb8e-5542-44c4-b147-38ed55f8f928","Handler":null,"Name":"","Endpoint":""}
Jul 09 15:02:54.148496 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:54.148476    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 15:02:54.148596 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:54.148516    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 15:02:54.225029 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:54.225002    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:54.225197 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:54.225138    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:02:55.225595 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:55.225397    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:55.226060 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:55.225443    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:55.226060 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:55.225678    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:02:55.226060 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:55.225762    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:02:55.383697 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:55.383646    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj" event={"ID":"1e46b888-5ce8-43e0-858c-5545b28e9d9c","Type":"ContainerStarted","Data":"ac446d5abdf6cde202e77c28b59dac71aa01511db379880b709e74415352121b"}
Jul 09 15:02:55.386616 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:55.386558    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:02:55.386978 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:55.386952    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb" event={"ID":"1c59bea2-03b5-4242-8f9a-9510da7d4eca","Type":"ContainerStarted","Data":"9bed0c86adc6136f12d95e5141cf6dc5e88eaee4940d38d50282edb295546297"}
Jul 09 15:02:55.403952 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:55.403894    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwppj" podStartSLOduration=7.217368265 podStartE2EDuration="25.403878304s" totalImagesPullingTime="18.186510039s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:55.403404278 +0000 UTC m=+25.736834247" watchObservedRunningTime="2026-07-09 15:02:55.403878304 +0000 UTC m=+25.737308268"
Jul 09 15:02:55.951347 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:55.951314    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-r6r56"
Jul 09 15:02:55.951982 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:55.951963    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-r6r56"
Jul 09 15:02:56.226134 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:56.226050    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:56.226726 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:56.226180    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:02:56.388950 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:56.388907    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-r6r56"
Jul 09 15:02:56.389280 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:56.389262    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-r6r56"
Jul 09 15:02:57.225370 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:57.225175    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:57.225551 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:57.225176    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:57.225551 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:57.225444    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:02:57.225661 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:57.225550    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:02:57.393877 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:57.393848    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:02:57.394643 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:57.394243    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb" event={"ID":"1c59bea2-03b5-4242-8f9a-9510da7d4eca","Type":"ContainerStarted","Data":"6aecbdd7f8ae281bc83f66395790dfceeeea3fe8fe8606310eb54d5365485676"}
Jul 09 15:02:57.394823 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:57.394799    2579 scope.go:117] "RemoveContainer" containerID="b19c576a655fbba5db8ac5219ba7514d699f2feb47292c8a836945362abd1de0"
Jul 09 15:02:58.225801 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:58.225769    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:58.226002 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:58.225947    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:02:58.400556 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:58.400529    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:02:58.401058 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:58.400966    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb" event={"ID":"1c59bea2-03b5-4242-8f9a-9510da7d4eca","Type":"ContainerStarted","Data":"95b7a2e4e66a728736c53c7c1ec712eaf62ee962045e1ccc9b40f9839eeea43d"}
Jul 09 15:02:58.401389 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:58.401353    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:58.401518 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:58.401426    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:58.401518 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:58.401460    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:58.418227 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:58.418199    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:58.418380 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:58.418308    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:02:58.442297 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:58.442242    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb" podStartSLOduration=8.567097575 podStartE2EDuration="28.442225902s" totalImagesPullingTime="19.875128327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:02:58.440591543 +0000 UTC m=+28.774021508" watchObservedRunningTime="2026-07-09 15:02:58.442225902 +0000 UTC m=+28.775655867"
Jul 09 15:02:58.932311 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:58.932264    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6ltzh"]
Jul 09 15:02:58.932533 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:58.932460    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:02:58.932601 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:58.932580    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:02:58.933170 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:58.933148    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qw9p7"]
Jul 09 15:02:58.933285 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:58.933251    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:02:58.933432 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:58.933347    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:02:58.943787 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:58.943746    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9swxx"]
Jul 09 15:02:58.943961 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:02:58.943914    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:02:58.944054 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:02:58.944034    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:03:00.229022 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:00.228994    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:03:00.229449 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:00.229105    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:03:01.225697 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:01.225454    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:03:01.225843 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:01.225454    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:03:01.225843 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:01.225735    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:03:01.225843 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:01.225800    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:03:01.411191 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:01.411161    2579 generic.go:356] "Generic (PLEG): container finished" podID="d4792ad3-2b68-4342-91e0-cc307d60e42a" containerID="49b7e7466ca974029196b4e17b807e98e2a42412ea0f8bc5168b37bf80e52950" exitCode=0
Jul 09 15:03:01.411791 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:01.411216    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8h8j" event={"ID":"d4792ad3-2b68-4342-91e0-cc307d60e42a","Type":"ContainerDied","Data":"49b7e7466ca974029196b4e17b807e98e2a42412ea0f8bc5168b37bf80e52950"}
Jul 09 15:03:02.228680 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:02.228656    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:03:02.228856 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:02.228779    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:03:02.906047 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:02.906004    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:03:02.906047 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:02.906058    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs\") pod \"network-metrics-daemon-9swxx\" (UID: \"5fb7d2e1-0cc8-4827-a59c-59487768f9d3\") " pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:03:02.906572 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:02.906157    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:03:02.906572 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:02.906163    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 15:03:02.906572 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:02.906214    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs podName:5fb7d2e1-0cc8-4827-a59c-59487768f9d3 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:34.906198891 +0000 UTC m=+65.239628842 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs") pod "network-metrics-daemon-9swxx" (UID: "5fb7d2e1-0cc8-4827-a59c-59487768f9d3") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:03:02.906572 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:02.906228    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret podName:a4e083b0-d118-451b-8655-30df5353d5ed nodeName:}" failed. No retries permitted until 2026-07-09 15:03:34.906221967 +0000 UTC m=+65.239651910 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret") pod "global-pull-secret-syncer-6ltzh" (UID: "a4e083b0-d118-451b-8655-30df5353d5ed") : object "kube-system"/"original-pull-secret" not registered
Jul 09 15:03:03.006365 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:03.006329    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82r5h\" (UniqueName: \"kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h\") pod \"network-check-target-qw9p7\" (UID: \"68d94daa-3588-4e94-be77-28ab021314f5\") " pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:03:03.006548 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:03.006480    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 15:03:03.006548 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:03.006497    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 15:03:03.006548 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:03.006507    2579 projected.go:196] Error preparing data for projected volume kube-api-access-82r5h for pod openshift-network-diagnostics/network-check-target-qw9p7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:03:03.006640 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:03.006562    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h podName:68d94daa-3588-4e94-be77-28ab021314f5 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:35.006545344 +0000 UTC m=+65.339975298 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-82r5h" (UniqueName: "kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h") pod "network-check-target-qw9p7" (UID: "68d94daa-3588-4e94-be77-28ab021314f5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:03:03.225213 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:03.225179    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:03:03.225372 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:03.225181    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:03:03.225372 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:03.225292    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6ltzh" podUID="a4e083b0-d118-451b-8655-30df5353d5ed"
Jul 09 15:03:03.225372 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:03.225354    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qw9p7" podUID="68d94daa-3588-4e94-be77-28ab021314f5"
Jul 09 15:03:04.228258 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:04.228225    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:03:04.228700 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:04.228353    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9swxx" podUID="5fb7d2e1-0cc8-4827-a59c-59487768f9d3"
Jul 09 15:03:04.266565 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:04.266536    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jrt9w_7ce9d4e7-f0cf-49b3-b925-12dc259bdd85/dns-node-resolver/0.log"
Jul 09 15:03:04.423480 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:04.423445    2579 generic.go:356] "Generic (PLEG): container finished" podID="d4792ad3-2b68-4342-91e0-cc307d60e42a" containerID="dac24c44c1de15819ce13d1b70fcc2ac735427e601fd8abe3f1d25444dacdc5e" exitCode=0
Jul 09 15:03:04.423718 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:04.423501    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8h8j" event={"ID":"d4792ad3-2b68-4342-91e0-cc307d60e42a","Type":"ContainerDied","Data":"dac24c44c1de15819ce13d1b70fcc2ac735427e601fd8abe3f1d25444dacdc5e"}
Jul 09 15:03:04.972454 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:04.972411    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-152.ec2.internal" event="NodeReady"
Jul 09 15:03:04.972654 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:04.972565    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 15:03:05.020709 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.020656    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-58f9d88667-gshlx"]
Jul 09 15:03:05.032494 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.032458    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.035287 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.035256    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mfwjc"]
Jul 09 15:03:05.036243 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.036213    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 09 15:03:05.036374 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.036296    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 09 15:03:05.036374 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.036302    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 09 15:03:05.036624 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.036608    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zbftx\""
Jul 09 15:03:05.042714 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.042679    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 09 15:03:05.045415 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.045393    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58f9d88667-gshlx"]
Jul 09 15:03:05.045521 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.045498    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.045521 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.045516    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-spbqk"]
Jul 09 15:03:05.048420 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.048398    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 15:03:05.048775 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.048754    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 15:03:05.048866 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.048832    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 15:03:05.048866 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.048860    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nt8wt\""
Jul 09 15:03:05.048989 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.048949    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 15:03:05.058040 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.058014    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mfwjc"]
Jul 09 15:03:05.058180 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.058128    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:05.061199 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.061009    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lcjqv\""
Jul 09 15:03:05.061491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.061468    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-spbqk"]
Jul 09 15:03:05.061647 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.061626    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 15:03:05.061935 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.061919    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 15:03:05.074889 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.074854    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q86vm"]
Jul 09 15:03:05.096873 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.096598    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q86vm"]
Jul 09 15:03:05.096873 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.096762    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q86vm"
Jul 09 15:03:05.100875 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.100841    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 15:03:05.101052 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.100929    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 15:03:05.101052 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.100950    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 15:03:05.101207 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.101123    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lqzhv\""
Jul 09 15:03:05.121543 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.121510    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f01b0b67-67ac-416c-9039-da018836dced-metrics-tls\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:05.121543 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.121547    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e597ee6-426e-4f2c-b0ad-e190913653b3-image-registry-private-configuration\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.121799 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.121572    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.121799 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.121595    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.121799 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.121645    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f01b0b67-67ac-416c-9039-da018836dced-tmp-dir\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:05.121799 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.121737    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqtll\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-kube-api-access-lqtll\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.121799 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.121784    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.121992 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.121812    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qcn8\" (UniqueName: \"kubernetes.io/projected/209f1dc4-865c-46b2-8ee8-83252469d366-kube-api-access-9qcn8\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.121992 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.121843    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-certificates\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.121992 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.121874    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f01b0b67-67ac-416c-9039-da018836dced-config-volume\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:05.121992 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.121901    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2zgj\" (UniqueName: \"kubernetes.io/projected/f01b0b67-67ac-416c-9039-da018836dced-kube-api-access-v2zgj\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:05.122128 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.122006    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e597ee6-426e-4f2c-b0ad-e190913653b3-trusted-ca\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.122128 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.122030    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/209f1dc4-865c-46b2-8ee8-83252469d366-data-volume\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.122128 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.122048    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e597ee6-426e-4f2c-b0ad-e190913653b3-ca-trust-extracted\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.122128 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.122072    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e597ee6-426e-4f2c-b0ad-e190913653b3-installation-pull-secrets\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.122128 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.122092    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-bound-sa-token\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.122128 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.122108    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/209f1dc4-865c-46b2-8ee8-83252469d366-crio-socket\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.222592 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.222550    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.222592 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.222594    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qcn8\" (UniqueName: \"kubernetes.io/projected/209f1dc4-865c-46b2-8ee8-83252469d366-kube-api-access-9qcn8\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.222876 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.222620    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-certificates\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.222876 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.222646    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f01b0b67-67ac-416c-9039-da018836dced-config-volume\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:05.222876 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.222667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2zgj\" (UniqueName: \"kubernetes.io/projected/f01b0b67-67ac-416c-9039-da018836dced-kube-api-access-v2zgj\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:05.222876 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.222707    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 15:03:05.222876 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.222755    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e597ee6-426e-4f2c-b0ad-e190913653b3-trusted-ca\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.222876 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.222795    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm podName:209f1dc4-865c-46b2-8ee8-83252469d366 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:05.722774106 +0000 UTC m=+36.056204049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mfwjc" (UID: "209f1dc4-865c-46b2-8ee8-83252469d366") : configmap "kube-rbac-proxy" not found
Jul 09 15:03:05.222876 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.222819    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/209f1dc4-865c-46b2-8ee8-83252469d366-data-volume\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.222876 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.222850    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj5wh\" (UniqueName: \"kubernetes.io/projected/0817217b-29ab-4c7d-8f05-874d0fc2b557-kube-api-access-xj5wh\") pod \"ingress-canary-q86vm\" (UID: \"0817217b-29ab-4c7d-8f05-874d0fc2b557\") " pod="openshift-ingress-canary/ingress-canary-q86vm"
Jul 09 15:03:05.223260 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.222884    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e597ee6-426e-4f2c-b0ad-e190913653b3-ca-trust-extracted\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.223260 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.222907    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e597ee6-426e-4f2c-b0ad-e190913653b3-installation-pull-secrets\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.223260 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.222931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-bound-sa-token\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.223260 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.222953    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/209f1dc4-865c-46b2-8ee8-83252469d366-crio-socket\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.223260 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.222983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f01b0b67-67ac-416c-9039-da018836dced-metrics-tls\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:05.223260 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.223012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e597ee6-426e-4f2c-b0ad-e190913653b3-image-registry-private-configuration\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.223260 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.223043    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.223260 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.223069    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.223260 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.223103    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f01b0b67-67ac-416c-9039-da018836dced-tmp-dir\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:05.223260 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.223128    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0817217b-29ab-4c7d-8f05-874d0fc2b557-cert\") pod \"ingress-canary-q86vm\" (UID: \"0817217b-29ab-4c7d-8f05-874d0fc2b557\") " pod="openshift-ingress-canary/ingress-canary-q86vm"
Jul 09 15:03:05.223260 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.223152    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 15:03:05.223260 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.223160    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqtll\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-kube-api-access-lqtll\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.223260 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.223198    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f01b0b67-67ac-416c-9039-da018836dced-metrics-tls podName:f01b0b67-67ac-416c-9039-da018836dced nodeName:}" failed. No retries permitted until 2026-07-09 15:03:05.723184213 +0000 UTC m=+36.056614170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f01b0b67-67ac-416c-9039-da018836dced-metrics-tls") pod "dns-default-spbqk" (UID: "f01b0b67-67ac-416c-9039-da018836dced") : secret "dns-default-metrics-tls" not found
Jul 09 15:03:05.223260 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.223233    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/209f1dc4-865c-46b2-8ee8-83252469d366-data-volume\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.223829 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.223326    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 15:03:05.223829 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.223348    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e597ee6-426e-4f2c-b0ad-e190913653b3-ca-trust-extracted\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.223829 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.223374    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-certificates\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.223829 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.223389    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls podName:209f1dc4-865c-46b2-8ee8-83252469d366 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:05.723371876 +0000 UTC m=+36.056801818 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mfwjc" (UID: "209f1dc4-865c-46b2-8ee8-83252469d366") : secret "insights-runtime-extractor-tls" not found
Jul 09 15:03:05.223829 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.223393    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f01b0b67-67ac-416c-9039-da018836dced-config-volume\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:05.223829 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.223477    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 15:03:05.223829 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.223489    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58f9d88667-gshlx: secret "image-registry-tls" not found
Jul 09 15:03:05.223829 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.223540    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls podName:4e597ee6-426e-4f2c-b0ad-e190913653b3 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:05.723525973 +0000 UTC m=+36.056955933 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls") pod "image-registry-58f9d88667-gshlx" (UID: "4e597ee6-426e-4f2c-b0ad-e190913653b3") : secret "image-registry-tls" not found
Jul 09 15:03:05.223829 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.223600    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/209f1dc4-865c-46b2-8ee8-83252469d366-crio-socket\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.223829 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.223630    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f01b0b67-67ac-416c-9039-da018836dced-tmp-dir\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:05.224181 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.223883    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e597ee6-426e-4f2c-b0ad-e190913653b3-trusted-ca\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.224993 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.224970    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:03:05.225205 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.225148    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:03:05.227750 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.227534    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e597ee6-426e-4f2c-b0ad-e190913653b3-image-registry-private-configuration\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.227901 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.227543    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e597ee6-426e-4f2c-b0ad-e190913653b3-installation-pull-secrets\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.227901 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.227591    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 15:03:05.227901 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.227591    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 15:03:05.227901 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.227614    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 15:03:05.227901 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.227615    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7htgt\""
Jul 09 15:03:05.232372 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.232347    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2zgj\" (UniqueName: \"kubernetes.io/projected/f01b0b67-67ac-416c-9039-da018836dced-kube-api-access-v2zgj\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:05.232719 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.232434    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qcn8\" (UniqueName: \"kubernetes.io/projected/209f1dc4-865c-46b2-8ee8-83252469d366-kube-api-access-9qcn8\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.232719 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.232597    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqtll\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-kube-api-access-lqtll\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.232719 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.232597    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-bound-sa-token\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.324252 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.324213    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj5wh\" (UniqueName: \"kubernetes.io/projected/0817217b-29ab-4c7d-8f05-874d0fc2b557-kube-api-access-xj5wh\") pod \"ingress-canary-q86vm\" (UID: \"0817217b-29ab-4c7d-8f05-874d0fc2b557\") " pod="openshift-ingress-canary/ingress-canary-q86vm"
Jul 09 15:03:05.324408 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.324304    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0817217b-29ab-4c7d-8f05-874d0fc2b557-cert\") pod \"ingress-canary-q86vm\" (UID: \"0817217b-29ab-4c7d-8f05-874d0fc2b557\") " pod="openshift-ingress-canary/ingress-canary-q86vm"
Jul 09 15:03:05.326823 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.326799    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0817217b-29ab-4c7d-8f05-874d0fc2b557-cert\") pod \"ingress-canary-q86vm\" (UID: \"0817217b-29ab-4c7d-8f05-874d0fc2b557\") " pod="openshift-ingress-canary/ingress-canary-q86vm"
Jul 09 15:03:05.343043 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.343007    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj5wh\" (UniqueName: \"kubernetes.io/projected/0817217b-29ab-4c7d-8f05-874d0fc2b557-kube-api-access-xj5wh\") pod \"ingress-canary-q86vm\" (UID: \"0817217b-29ab-4c7d-8f05-874d0fc2b557\") " pod="openshift-ingress-canary/ingress-canary-q86vm"
Jul 09 15:03:05.409258 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.409222    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q86vm"
Jul 09 15:03:05.436699 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.436667    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ncj7q_a973d8db-5cd4-490c-af42-c496f3ced07d/node-ca/0.log"
Jul 09 15:03:05.722874 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.722845    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q86vm"]
Jul 09 15:03:05.727137 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.727092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f01b0b67-67ac-416c-9039-da018836dced-metrics-tls\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:05.727268 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.727142    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:05.727268 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.727172    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.727268 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:05.727216    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:05.727443 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.727271    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 15:03:05.727443 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.727326    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 15:03:05.727443 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.727346    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f01b0b67-67ac-416c-9039-da018836dced-metrics-tls podName:f01b0b67-67ac-416c-9039-da018836dced nodeName:}" failed. No retries permitted until 2026-07-09 15:03:06.727324048 +0000 UTC m=+37.060754012 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f01b0b67-67ac-416c-9039-da018836dced-metrics-tls") pod "dns-default-spbqk" (UID: "f01b0b67-67ac-416c-9039-da018836dced") : secret "dns-default-metrics-tls" not found
Jul 09 15:03:05.727443 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.727349    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 15:03:05.727443 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.727379    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm podName:209f1dc4-865c-46b2-8ee8-83252469d366 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:06.727362597 +0000 UTC m=+37.060792545 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mfwjc" (UID: "209f1dc4-865c-46b2-8ee8-83252469d366") : configmap "kube-rbac-proxy" not found
Jul 09 15:03:05.727443 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.727404    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls podName:209f1dc4-865c-46b2-8ee8-83252469d366 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:06.727388759 +0000 UTC m=+37.060818726 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mfwjc" (UID: "209f1dc4-865c-46b2-8ee8-83252469d366") : secret "insights-runtime-extractor-tls" not found
Jul 09 15:03:05.727443 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.727412    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 15:03:05.727443 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.727432    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58f9d88667-gshlx: secret "image-registry-tls" not found
Jul 09 15:03:05.727801 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:05.727491    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls podName:4e597ee6-426e-4f2c-b0ad-e190913653b3 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:06.727475047 +0000 UTC m=+37.060904990 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls") pod "image-registry-58f9d88667-gshlx" (UID: "4e597ee6-426e-4f2c-b0ad-e190913653b3") : secret "image-registry-tls" not found
Jul 09 15:03:05.772615 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:03:05.772575    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0817217b_29ab_4c7d_8f05_874d0fc2b557.slice/crio-616576c21d6567824a4854da471e94d7a2bfb37ae661439ef28b4f8f5ab91a53 WatchSource:0}: Error finding container 616576c21d6567824a4854da471e94d7a2bfb37ae661439ef28b4f8f5ab91a53: Status 404 returned error can't find the container with id 616576c21d6567824a4854da471e94d7a2bfb37ae661439ef28b4f8f5ab91a53
Jul 09 15:03:06.225385 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:06.225348    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:03:06.228397 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:06.228372    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 15:03:06.228622 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:06.228608    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z6fx5\""
Jul 09 15:03:06.430377 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:06.430344    2579 generic.go:356] "Generic (PLEG): container finished" podID="d4792ad3-2b68-4342-91e0-cc307d60e42a" containerID="309d85476f132b8e43511bd44ff8c763af090f9d4c79334d7ba0492bbd09a7d2" exitCode=0
Jul 09 15:03:06.430819 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:06.430434    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8h8j" event={"ID":"d4792ad3-2b68-4342-91e0-cc307d60e42a","Type":"ContainerDied","Data":"309d85476f132b8e43511bd44ff8c763af090f9d4c79334d7ba0492bbd09a7d2"}
Jul 09 15:03:06.431788 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:06.431611    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q86vm" event={"ID":"0817217b-29ab-4c7d-8f05-874d0fc2b557","Type":"ContainerStarted","Data":"616576c21d6567824a4854da471e94d7a2bfb37ae661439ef28b4f8f5ab91a53"}
Jul 09 15:03:06.739416 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:06.739381    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f01b0b67-67ac-416c-9039-da018836dced-metrics-tls\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:06.739416 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:06.739419    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:06.739704 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:06.739439    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:06.739704 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:06.739478    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:06.739704 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:06.739533    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 15:03:06.739704 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:06.739544    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 15:03:06.739704 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:06.739554    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 15:03:06.739704 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:06.739562    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58f9d88667-gshlx: secret "image-registry-tls" not found
Jul 09 15:03:06.739704 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:06.739585    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 15:03:06.739704 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:06.739633    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f01b0b67-67ac-416c-9039-da018836dced-metrics-tls podName:f01b0b67-67ac-416c-9039-da018836dced nodeName:}" failed. No retries permitted until 2026-07-09 15:03:08.739592249 +0000 UTC m=+39.073022195 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f01b0b67-67ac-416c-9039-da018836dced-metrics-tls") pod "dns-default-spbqk" (UID: "f01b0b67-67ac-416c-9039-da018836dced") : secret "dns-default-metrics-tls" not found
Jul 09 15:03:06.739704 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:06.739648    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls podName:209f1dc4-865c-46b2-8ee8-83252469d366 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:08.73964208 +0000 UTC m=+39.073072023 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mfwjc" (UID: "209f1dc4-865c-46b2-8ee8-83252469d366") : secret "insights-runtime-extractor-tls" not found
Jul 09 15:03:06.739704 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:06.739661    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls podName:4e597ee6-426e-4f2c-b0ad-e190913653b3 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:08.73965385 +0000 UTC m=+39.073083792 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls") pod "image-registry-58f9d88667-gshlx" (UID: "4e597ee6-426e-4f2c-b0ad-e190913653b3") : secret "image-registry-tls" not found
Jul 09 15:03:06.739704 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:06.739675    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm podName:209f1dc4-865c-46b2-8ee8-83252469d366 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:08.739666186 +0000 UTC m=+39.073096128 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mfwjc" (UID: "209f1dc4-865c-46b2-8ee8-83252469d366") : configmap "kube-rbac-proxy" not found
Jul 09 15:03:08.759085 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:08.759041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:08.759558 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:08.759100    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:08.759558 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:08.759224    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 15:03:08.759558 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:08.759232    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 15:03:08.759558 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:08.759255    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58f9d88667-gshlx: secret "image-registry-tls" not found
Jul 09 15:03:08.759558 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:08.759264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:08.759558 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:08.759286    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls podName:209f1dc4-865c-46b2-8ee8-83252469d366 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:12.759266982 +0000 UTC m=+43.092696930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mfwjc" (UID: "209f1dc4-865c-46b2-8ee8-83252469d366") : secret "insights-runtime-extractor-tls" not found
Jul 09 15:03:08.759558 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:08.759310    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls podName:4e597ee6-426e-4f2c-b0ad-e190913653b3 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:12.759300756 +0000 UTC m=+43.092730699 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls") pod "image-registry-58f9d88667-gshlx" (UID: "4e597ee6-426e-4f2c-b0ad-e190913653b3") : secret "image-registry-tls" not found
Jul 09 15:03:08.759558 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:08.759336    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 15:03:08.759558 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:08.759384    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm podName:209f1dc4-865c-46b2-8ee8-83252469d366 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:12.759367904 +0000 UTC m=+43.092797851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mfwjc" (UID: "209f1dc4-865c-46b2-8ee8-83252469d366") : configmap "kube-rbac-proxy" not found
Jul 09 15:03:08.759558 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:08.759473    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f01b0b67-67ac-416c-9039-da018836dced-metrics-tls\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:08.760080 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:08.759578    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 15:03:08.760080 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:08.759616    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f01b0b67-67ac-416c-9039-da018836dced-metrics-tls podName:f01b0b67-67ac-416c-9039-da018836dced nodeName:}" failed. No retries permitted until 2026-07-09 15:03:12.759604687 +0000 UTC m=+43.093034630 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f01b0b67-67ac-416c-9039-da018836dced-metrics-tls") pod "dns-default-spbqk" (UID: "f01b0b67-67ac-416c-9039-da018836dced") : secret "dns-default-metrics-tls" not found
Jul 09 15:03:09.440979 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:09.440787    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q86vm" event={"ID":"0817217b-29ab-4c7d-8f05-874d0fc2b557","Type":"ContainerStarted","Data":"0c74e484a8918f379394d333ca778ee372eaf9fb464dbad63b262f95a89dc37e"}
Jul 09 15:03:09.461460 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:09.461408    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q86vm" podStartSLOduration=1.079235194 podStartE2EDuration="4.461392501s" totalImagesPullingTime="3.382157307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:09.460521105 +0000 UTC m=+39.793951073" watchObservedRunningTime="2026-07-09 15:03:09.461392501 +0000 UTC m=+39.794822465"
Jul 09 15:03:12.451900 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:12.451865    2579 generic.go:356] "Generic (PLEG): container finished" podID="d4792ad3-2b68-4342-91e0-cc307d60e42a" containerID="81c4916894911e34123dd012269d832678677ee06487d1d13cfc8dea3333b786" exitCode=0
Jul 09 15:03:12.452431 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:12.451922    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8h8j" event={"ID":"d4792ad3-2b68-4342-91e0-cc307d60e42a","Type":"ContainerDied","Data":"81c4916894911e34123dd012269d832678677ee06487d1d13cfc8dea3333b786"}
Jul 09 15:03:12.795017 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:12.794905    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f01b0b67-67ac-416c-9039-da018836dced-metrics-tls\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:12.795017 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:12.794959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:12.795017 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:12.794988    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:12.795289 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:12.795030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:12.795289 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:12.795144    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 15:03:12.795289 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:12.795160    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 15:03:12.795289 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:12.795230    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm podName:209f1dc4-865c-46b2-8ee8-83252469d366 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:20.795209818 +0000 UTC m=+51.128639783 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mfwjc" (UID: "209f1dc4-865c-46b2-8ee8-83252469d366") : configmap "kube-rbac-proxy" not found
Jul 09 15:03:12.795289 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:03:12.795250    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls podName:209f1dc4-865c-46b2-8ee8-83252469d366 nodeName:}" failed. No retries permitted until 2026-07-09 15:03:20.795240759 +0000 UTC m=+51.128670708 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mfwjc" (UID: "209f1dc4-865c-46b2-8ee8-83252469d366") : secret "insights-runtime-extractor-tls" not found
Jul 09 15:03:12.798895 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:12.798869    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f01b0b67-67ac-416c-9039-da018836dced-metrics-tls\") pod \"dns-default-spbqk\" (UID: \"f01b0b67-67ac-416c-9039-da018836dced\") " pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:12.798998 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:12.798962    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls\") pod \"image-registry-58f9d88667-gshlx\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") " pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:12.842043 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:12.842005    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:12.868094 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:12.868061    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:13.003358 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:13.003327    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58f9d88667-gshlx"]
Jul 09 15:03:13.006238 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:03:13.006210    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e597ee6_426e_4f2c_b0ad_e190913653b3.slice/crio-c6e9eeb0e65feb8bd634cf0a62cfb98d327a01071db35408e771ceccade70d4c WatchSource:0}: Error finding container c6e9eeb0e65feb8bd634cf0a62cfb98d327a01071db35408e771ceccade70d4c: Status 404 returned error can't find the container with id c6e9eeb0e65feb8bd634cf0a62cfb98d327a01071db35408e771ceccade70d4c
Jul 09 15:03:13.033023 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:13.032989    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-spbqk"]
Jul 09 15:03:13.038457 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:03:13.038415    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf01b0b67_67ac_416c_9039_da018836dced.slice/crio-b43ebe0ecc5c2877a1fbff10042bf17da963145d42729d076b5e1fc3951d8657 WatchSource:0}: Error finding container b43ebe0ecc5c2877a1fbff10042bf17da963145d42729d076b5e1fc3951d8657: Status 404 returned error can't find the container with id b43ebe0ecc5c2877a1fbff10042bf17da963145d42729d076b5e1fc3951d8657
Jul 09 15:03:13.455608 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:13.455572    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-spbqk" event={"ID":"f01b0b67-67ac-416c-9039-da018836dced","Type":"ContainerStarted","Data":"b43ebe0ecc5c2877a1fbff10042bf17da963145d42729d076b5e1fc3951d8657"}
Jul 09 15:03:13.458510 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:13.458477    2579 generic.go:356] "Generic (PLEG): container finished" podID="d4792ad3-2b68-4342-91e0-cc307d60e42a" containerID="19326308f67157a8bd482946ea725055be04b50e00dd07a8cbcd9f6910470ca7" exitCode=0
Jul 09 15:03:13.458621 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:13.458562    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8h8j" event={"ID":"d4792ad3-2b68-4342-91e0-cc307d60e42a","Type":"ContainerDied","Data":"19326308f67157a8bd482946ea725055be04b50e00dd07a8cbcd9f6910470ca7"}
Jul 09 15:03:13.460079 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:13.460056    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58f9d88667-gshlx" event={"ID":"4e597ee6-426e-4f2c-b0ad-e190913653b3","Type":"ContainerStarted","Data":"7c65e3dfcb2820acb89b96d1dbed2e68a05f92f66448731a6dfd2459fbd31386"}
Jul 09 15:03:13.460154 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:13.460093    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58f9d88667-gshlx" event={"ID":"4e597ee6-426e-4f2c-b0ad-e190913653b3","Type":"ContainerStarted","Data":"c6e9eeb0e65feb8bd634cf0a62cfb98d327a01071db35408e771ceccade70d4c"}
Jul 09 15:03:13.460219 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:13.460202    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:13.505472 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:13.505413    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-58f9d88667-gshlx" podStartSLOduration=30.505392719 podStartE2EDuration="30.505392719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:13.504303581 +0000 UTC m=+43.837733550" watchObservedRunningTime="2026-07-09 15:03:13.505392719 +0000 UTC m=+43.838822684"
Jul 09 15:03:14.466065 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:14.466020    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8h8j" event={"ID":"d4792ad3-2b68-4342-91e0-cc307d60e42a","Type":"ContainerStarted","Data":"675fd07c1010e9e71c52a15457386a61efbb0246ac2c8a35690160babdbc944a"}
Jul 09 15:03:14.496252 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:14.496176    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-t8h8j" podStartSLOduration=7.5742838169999995 podStartE2EDuration="44.496154746s" totalImagesPullingTime="36.921870929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:14.493109694 +0000 UTC m=+44.826539659" watchObservedRunningTime="2026-07-09 15:03:14.496154746 +0000 UTC m=+44.829584714"
Jul 09 15:03:16.473218 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:16.473180    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-spbqk" event={"ID":"f01b0b67-67ac-416c-9039-da018836dced","Type":"ContainerStarted","Data":"3bfceef8124ffd14953dc219225235d5a4d384e7ca0027a99c1a7e153e73814e"}
Jul 09 15:03:16.473702 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:16.473241    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-spbqk" event={"ID":"f01b0b67-67ac-416c-9039-da018836dced","Type":"ContainerStarted","Data":"fe295eda0a586e87aed047deff5f0868f0741d02a9d11b3fa41411acbfe7ecb6"}
Jul 09 15:03:16.473702 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:16.473318    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:16.494045 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:16.493990    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-spbqk" podStartSLOduration=8.989050644 podStartE2EDuration="11.493974156s" totalImagesPullingTime="2.504923512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:16.492876523 +0000 UTC m=+46.826306489" watchObservedRunningTime="2026-07-09 15:03:16.493974156 +0000 UTC m=+46.827404120"
Jul 09 15:03:20.859839 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:20.859805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:20.860208 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:20.859908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:20.860339 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:20.860310    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/209f1dc4-865c-46b2-8ee8-83252469d366-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:20.862043 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:20.862017    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/209f1dc4-865c-46b2-8ee8-83252469d366-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfwjc\" (UID: \"209f1dc4-865c-46b2-8ee8-83252469d366\") " pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:20.953957 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:20.953919    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mfwjc"
Jul 09 15:03:21.094284 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:21.094251    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mfwjc"]
Jul 09 15:03:21.097195 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:03:21.097153    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod209f1dc4_865c_46b2_8ee8_83252469d366.slice/crio-af042caa12054013a0e60914d15b59dd5262f3e5ec2e011d9aaa894a58745fb8 WatchSource:0}: Error finding container af042caa12054013a0e60914d15b59dd5262f3e5ec2e011d9aaa894a58745fb8: Status 404 returned error can't find the container with id af042caa12054013a0e60914d15b59dd5262f3e5ec2e011d9aaa894a58745fb8
Jul 09 15:03:21.487054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:21.487020    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mfwjc" event={"ID":"209f1dc4-865c-46b2-8ee8-83252469d366","Type":"ContainerStarted","Data":"af042caa12054013a0e60914d15b59dd5262f3e5ec2e011d9aaa894a58745fb8"}
Jul 09 15:03:24.446933 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.446900    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696449858b-2kngq"]
Jul 09 15:03:24.461667 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.461610    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"]
Jul 09 15:03:24.461886 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.461860    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696449858b-2kngq"
Jul 09 15:03:24.468902 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.466006    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-5frnm\""
Jul 09 15:03:24.468902 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.466331    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 09 15:03:24.468902 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.466638    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 09 15:03:24.468902 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.468233    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"]
Jul 09 15:03:24.468902 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.468819    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 09 15:03:24.470557 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.470166    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 09 15:03:24.472144 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.472119    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"
Jul 09 15:03:24.474401 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.474378    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-58f9d88667-gshlx"]
Jul 09 15:03:24.474633 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.474599    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696449858b-2kngq"]
Jul 09 15:03:24.474733 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.474643    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"]
Jul 09 15:03:24.474733 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.474668    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"]
Jul 09 15:03:24.474733 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.474488    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.475024 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.474924    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 09 15:03:24.478129 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.478107    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 09 15:03:24.478129 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.478121    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 09 15:03:24.478402 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.478387    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 09 15:03:24.478477 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.478409    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 09 15:03:24.484416 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.484394    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-hub\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.484530 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.484422    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-additional-service-ca\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.484530 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.484461    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cvwf\" (UniqueName: \"kubernetes.io/projected/ebb93425-b316-42eb-bc1a-79b69decd42a-kube-api-access-5cvwf\") pod \"managed-serviceaccount-addon-agent-696449858b-2kngq\" (UID: \"ebb93425-b316-42eb-bc1a-79b69decd42a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696449858b-2kngq"
Jul 09 15:03:24.484625 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.484575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.484625 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.484613    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.484762 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.484645    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e90e78b-8a0d-49b9-924b-b8f468a171cf-tmp\") pod \"klusterlet-addon-workmgr-5b9844cd4-d8lht\" (UID: \"9e90e78b-8a0d-49b9-924b-b8f468a171cf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"
Jul 09 15:03:24.484762 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.484725    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sx2n\" (UniqueName: \"kubernetes.io/projected/9e90e78b-8a0d-49b9-924b-b8f468a171cf-kube-api-access-9sx2n\") pod \"klusterlet-addon-workmgr-5b9844cd4-d8lht\" (UID: \"9e90e78b-8a0d-49b9-924b-b8f468a171cf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"
Jul 09 15:03:24.484863 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.484758    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-ca\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.484863 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.484824    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9e90e78b-8a0d-49b9-924b-b8f468a171cf-klusterlet-config\") pod \"klusterlet-addon-workmgr-5b9844cd4-d8lht\" (UID: \"9e90e78b-8a0d-49b9-924b-b8f468a171cf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"
Jul 09 15:03:24.484863 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.484851    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88k5s\" (UniqueName: \"kubernetes.io/projected/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-kube-api-access-88k5s\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.484993 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.484890    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ebb93425-b316-42eb-bc1a-79b69decd42a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-696449858b-2kngq\" (UID: \"ebb93425-b316-42eb-bc1a-79b69decd42a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696449858b-2kngq"
Jul 09 15:03:24.550158 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.550127    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56b7c85567-zdl9d"]
Jul 09 15:03:24.553398 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.553373    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.567408 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.567359    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56b7c85567-zdl9d"]
Jul 09 15:03:24.585846 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.585815    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-registry-certificates\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.585846 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.585848    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-ca-trust-extracted\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.586077 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.585871    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sx2n\" (UniqueName: \"kubernetes.io/projected/9e90e78b-8a0d-49b9-924b-b8f468a171cf-kube-api-access-9sx2n\") pod \"klusterlet-addon-workmgr-5b9844cd4-d8lht\" (UID: \"9e90e78b-8a0d-49b9-924b-b8f468a171cf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"
Jul 09 15:03:24.586077 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.585889    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-ca\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.586077 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.585950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-installation-pull-secrets\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.586077 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.585995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9e90e78b-8a0d-49b9-924b-b8f468a171cf-klusterlet-config\") pod \"klusterlet-addon-workmgr-5b9844cd4-d8lht\" (UID: \"9e90e78b-8a0d-49b9-924b-b8f468a171cf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"
Jul 09 15:03:24.586077 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.586013    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88k5s\" (UniqueName: \"kubernetes.io/projected/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-kube-api-access-88k5s\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.586077 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.586049    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ebb93425-b316-42eb-bc1a-79b69decd42a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-696449858b-2kngq\" (UID: \"ebb93425-b316-42eb-bc1a-79b69decd42a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696449858b-2kngq"
Jul 09 15:03:24.586376 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.586084    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-image-registry-private-configuration\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.586376 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.586111    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-bound-sa-token\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.586376 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.586144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-hub\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.586376 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.586269    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-additional-service-ca\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.586586 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.586469    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cvwf\" (UniqueName: \"kubernetes.io/projected/ebb93425-b316-42eb-bc1a-79b69decd42a-kube-api-access-5cvwf\") pod \"managed-serviceaccount-addon-agent-696449858b-2kngq\" (UID: \"ebb93425-b316-42eb-bc1a-79b69decd42a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696449858b-2kngq"
Jul 09 15:03:24.586586 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.586506    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-registry-tls\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.586586 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.586538    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpcxc\" (UniqueName: \"kubernetes.io/projected/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-kube-api-access-xpcxc\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.586836 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.586587    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.586836 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.586623    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.586836 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.586659    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e90e78b-8a0d-49b9-924b-b8f468a171cf-tmp\") pod \"klusterlet-addon-workmgr-5b9844cd4-d8lht\" (UID: \"9e90e78b-8a0d-49b9-924b-b8f468a171cf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"
Jul 09 15:03:24.586836 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.586711    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-trusted-ca\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.587051 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.586999    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-additional-service-ca\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.587395 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.587368    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9e90e78b-8a0d-49b9-924b-b8f468a171cf-tmp\") pod \"klusterlet-addon-workmgr-5b9844cd4-d8lht\" (UID: \"9e90e78b-8a0d-49b9-924b-b8f468a171cf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"
Jul 09 15:03:24.589048 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.588932    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-ca\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.589048 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.589010    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-hub\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.589253 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.589210    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9e90e78b-8a0d-49b9-924b-b8f468a171cf-klusterlet-config\") pod \"klusterlet-addon-workmgr-5b9844cd4-d8lht\" (UID: \"9e90e78b-8a0d-49b9-924b-b8f468a171cf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"
Jul 09 15:03:24.589642 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.589624    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.589789 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.589767    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ebb93425-b316-42eb-bc1a-79b69decd42a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-696449858b-2kngq\" (UID: \"ebb93425-b316-42eb-bc1a-79b69decd42a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696449858b-2kngq"
Jul 09 15:03:24.590607 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.590556    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.603072 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.603039    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88k5s\" (UniqueName: \"kubernetes.io/projected/de86d299-33e3-44a7-a3f9-0f8904d7c2ab-kube-api-access-88k5s\") pod \"cluster-proxy-proxy-agent-569ddc4ff6-77l7f\" (UID: \"de86d299-33e3-44a7-a3f9-0f8904d7c2ab\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.603072 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.603071    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cvwf\" (UniqueName: \"kubernetes.io/projected/ebb93425-b316-42eb-bc1a-79b69decd42a-kube-api-access-5cvwf\") pod \"managed-serviceaccount-addon-agent-696449858b-2kngq\" (UID: \"ebb93425-b316-42eb-bc1a-79b69decd42a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696449858b-2kngq"
Jul 09 15:03:24.603254 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.603049    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sx2n\" (UniqueName: \"kubernetes.io/projected/9e90e78b-8a0d-49b9-924b-b8f468a171cf-kube-api-access-9sx2n\") pod \"klusterlet-addon-workmgr-5b9844cd4-d8lht\" (UID: \"9e90e78b-8a0d-49b9-924b-b8f468a171cf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"
Jul 09 15:03:24.687128 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.687090    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-trusted-ca\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.687128 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.687132    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-registry-certificates\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.687405 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.687155    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-ca-trust-extracted\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.687405 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.687187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-installation-pull-secrets\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.687405 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.687359    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-image-registry-private-configuration\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.687575 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.687411    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-bound-sa-token\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.687575 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.687452    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-registry-tls\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.687575 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.687480    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpcxc\" (UniqueName: \"kubernetes.io/projected/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-kube-api-access-xpcxc\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.687798 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.687594    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-ca-trust-extracted\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.688081 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.688034    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-registry-certificates\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.688320 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.688297    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-trusted-ca\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.690280 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.690242    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-image-registry-private-configuration\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.690399 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.690337    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-registry-tls\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.690399 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.690380    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-installation-pull-secrets\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.707146 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.707076    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-bound-sa-token\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.707616 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.707587    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpcxc\" (UniqueName: \"kubernetes.io/projected/1ace1ee5-8e0c-4ff2-9cda-03c238288a9b-kube-api-access-xpcxc\") pod \"image-registry-56b7c85567-zdl9d\" (UID: \"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b\") " pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:24.790601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.790563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696449858b-2kngq"
Jul 09 15:03:24.798107 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.798065    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"
Jul 09 15:03:24.803384 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.803359    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"
Jul 09 15:03:24.879011 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:24.878966    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:25.944217 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:25.944183    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56b7c85567-zdl9d"]
Jul 09 15:03:25.970612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:25.970579    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"]
Jul 09 15:03:25.979945 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:03:25.979908    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e90e78b_8a0d_49b9_924b_b8f468a171cf.slice/crio-de00f4f4e310617649f63d8428a7af43ddf8e794b4a57d0035c95a4c0feb0b18 WatchSource:0}: Error finding container de00f4f4e310617649f63d8428a7af43ddf8e794b4a57d0035c95a4c0feb0b18: Status 404 returned error can't find the container with id de00f4f4e310617649f63d8428a7af43ddf8e794b4a57d0035c95a4c0feb0b18
Jul 09 15:03:25.981863 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:25.981771    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696449858b-2kngq"]
Jul 09 15:03:25.985588 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:03:25.985566    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podebb93425_b316_42eb_bc1a_79b69decd42a.slice/crio-03e2cc1a08985eaaaa7cce3034f3e8e8c763d238321bbe8882aed1b5f014fe5e WatchSource:0}: Error finding container 03e2cc1a08985eaaaa7cce3034f3e8e8c763d238321bbe8882aed1b5f014fe5e: Status 404 returned error can't find the container with id 03e2cc1a08985eaaaa7cce3034f3e8e8c763d238321bbe8882aed1b5f014fe5e
Jul 09 15:03:26.004965 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:26.004810    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f"]
Jul 09 15:03:26.007401 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:03:26.007378    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde86d299_33e3_44a7_a3f9_0f8904d7c2ab.slice/crio-cdbb5211e9f79517ff9a080a08dfc8b2b341436b58cac08afeb25c82f12ca8b9 WatchSource:0}: Error finding container cdbb5211e9f79517ff9a080a08dfc8b2b341436b58cac08afeb25c82f12ca8b9: Status 404 returned error can't find the container with id cdbb5211e9f79517ff9a080a08dfc8b2b341436b58cac08afeb25c82f12ca8b9
Jul 09 15:03:26.477582 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:26.477544    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-spbqk"
Jul 09 15:03:26.506298 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:26.506264    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mfwjc" event={"ID":"209f1dc4-865c-46b2-8ee8-83252469d366","Type":"ContainerStarted","Data":"a4425f3de84b11e06b0eb14ebcfafda0dae88228efca2f45695be924b1171a42"}
Jul 09 15:03:26.507900 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:26.507866    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56b7c85567-zdl9d" event={"ID":"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b","Type":"ContainerStarted","Data":"e7561a1a6a6422cb8a51ac2e384efc0c687bfe266a7149bee717a950f976cec7"}
Jul 09 15:03:26.508034 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:26.507919    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56b7c85567-zdl9d" event={"ID":"1ace1ee5-8e0c-4ff2-9cda-03c238288a9b","Type":"ContainerStarted","Data":"ff5d369da50fce25da9d87db2bf2fb4d49804256997115384a81f5013dd1d334"}
Jul 09 15:03:26.508034 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:26.507978    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:26.509559 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:26.509533    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f" event={"ID":"de86d299-33e3-44a7-a3f9-0f8904d7c2ab","Type":"ContainerStarted","Data":"cdbb5211e9f79517ff9a080a08dfc8b2b341436b58cac08afeb25c82f12ca8b9"}
Jul 09 15:03:26.510973 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:26.510949    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696449858b-2kngq" event={"ID":"ebb93425-b316-42eb-bc1a-79b69decd42a","Type":"ContainerStarted","Data":"03e2cc1a08985eaaaa7cce3034f3e8e8c763d238321bbe8882aed1b5f014fe5e"}
Jul 09 15:03:26.512111 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:26.512082    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht" event={"ID":"9e90e78b-8a0d-49b9-924b-b8f468a171cf","Type":"ContainerStarted","Data":"de00f4f4e310617649f63d8428a7af43ddf8e794b4a57d0035c95a4c0feb0b18"}
Jul 09 15:03:26.535226 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:26.535156    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56b7c85567-zdl9d" podStartSLOduration=2.5351348160000002 podStartE2EDuration="2.535134816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:26.534107514 +0000 UTC m=+56.867537478" watchObservedRunningTime="2026-07-09 15:03:26.535134816 +0000 UTC m=+56.868564760"
Jul 09 15:03:29.530193 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:29.530150    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mfwjc" event={"ID":"209f1dc4-865c-46b2-8ee8-83252469d366","Type":"ContainerStarted","Data":"6a80d19c8d69375ff3cc1eadcb6553bc1bee0f463fd92881f18ed12163d272af"}
Jul 09 15:03:30.418529 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:30.418493    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n8ctb"
Jul 09 15:03:33.547195 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:33.547165    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696449858b-2kngq" event={"ID":"ebb93425-b316-42eb-bc1a-79b69decd42a","Type":"ContainerStarted","Data":"dc8c5212b92d96713571151a854f7f147d403ca0a20d06dfc0b8376b3ba038de"}
Jul 09 15:03:33.569224 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:33.569148    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-696449858b-2kngq" podStartSLOduration=6.108338627 podStartE2EDuration="9.56912412s" totalImagesPullingTime="3.460785493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:33.567282525 +0000 UTC m=+63.900712490" watchObservedRunningTime="2026-07-09 15:03:33.56912412 +0000 UTC m=+63.902554087"
Jul 09 15:03:34.480507 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.480476    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:34.552391 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.552353    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht" event={"ID":"9e90e78b-8a0d-49b9-924b-b8f468a171cf","Type":"ContainerStarted","Data":"747cf8d4cba45c1edee0b61058f3dd1b6219192a8a15202b6b5c4d48948c1f30"}
Jul 09 15:03:34.552887 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.552418    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"
Jul 09 15:03:34.554520 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.554494    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht"
Jul 09 15:03:34.554903 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.554878    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mfwjc" event={"ID":"209f1dc4-865c-46b2-8ee8-83252469d366","Type":"ContainerStarted","Data":"435a24c28ea8602a5886d55bae8d866aa30dabb168093cf03289990a2a4147c3"}
Jul 09 15:03:34.557179 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.557151    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f" event={"ID":"de86d299-33e3-44a7-a3f9-0f8904d7c2ab","Type":"ContainerStarted","Data":"17fd1798194d145621765f6aff81f5b0b8a17037e04b3cf8503a21ea2a9e7100"}
Jul 09 15:03:34.557291 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.557194    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f" event={"ID":"de86d299-33e3-44a7-a3f9-0f8904d7c2ab","Type":"ContainerStarted","Data":"7013ac61bce521f8f69b4c0a6740dad729d2132094905e5a6c35e7d0c8464199"}
Jul 09 15:03:34.557291 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.557213    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f" event={"ID":"de86d299-33e3-44a7-a3f9-0f8904d7c2ab","Type":"ContainerStarted","Data":"b985eda8eb5a2cf2197f4e30cb222897251f680141205e26d12a3f5f70885551"}
Jul 09 15:03:34.571034 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.570984    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b9844cd4-d8lht" podStartSLOduration=3.077120245 podStartE2EDuration="10.570965525s" totalImagesPullingTime="7.49384528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:34.570219841 +0000 UTC m=+64.903649806" watchObservedRunningTime="2026-07-09 15:03:34.570965525 +0000 UTC m=+64.904395490"
Jul 09 15:03:34.590827 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.590777    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mfwjc" podStartSLOduration=17.395029447 podStartE2EDuration="29.590759312s" totalImagesPullingTime="12.195729865s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:03:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:34.590365293 +0000 UTC m=+64.923795258" watchObservedRunningTime="2026-07-09 15:03:34.590759312 +0000 UTC m=+64.924189276"
Jul 09 15:03:34.627783 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.627723    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-569ddc4ff6-77l7f" podStartSLOduration=3.158002308 podStartE2EDuration="10.627704129s" totalImagesPullingTime="7.469701821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:34.627218449 +0000 UTC m=+64.960648416" watchObservedRunningTime="2026-07-09 15:03:34.627704129 +0000 UTC m=+64.961134084"
Jul 09 15:03:34.980328 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.980286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:03:34.980527 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.980391    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs\") pod \"network-metrics-daemon-9swxx\" (UID: \"5fb7d2e1-0cc8-4827-a59c-59487768f9d3\") " pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:03:34.983203 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.983185    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 15:03:34.983284 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.983237    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 15:03:34.992962 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.992928    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4e083b0-d118-451b-8655-30df5353d5ed-original-pull-secret\") pod \"global-pull-secret-syncer-6ltzh\" (UID: \"a4e083b0-d118-451b-8655-30df5353d5ed\") " pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:03:34.993061 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:34.992970    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fb7d2e1-0cc8-4827-a59c-59487768f9d3-metrics-certs\") pod \"network-metrics-daemon-9swxx\" (UID: \"5fb7d2e1-0cc8-4827-a59c-59487768f9d3\") " pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:03:35.043235 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:35.043201    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z6fx5\""
Jul 09 15:03:35.049737 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:35.049714    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9swxx"
Jul 09 15:03:35.081621 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:35.081578    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82r5h\" (UniqueName: \"kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h\") pod \"network-check-target-qw9p7\" (UID: \"68d94daa-3588-4e94-be77-28ab021314f5\") " pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:03:35.084321 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:35.084296    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 15:03:35.095315 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:35.095288    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 15:03:35.105080 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:35.105057    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82r5h\" (UniqueName: \"kubernetes.io/projected/68d94daa-3588-4e94-be77-28ab021314f5-kube-api-access-82r5h\") pod \"network-check-target-qw9p7\" (UID: \"68d94daa-3588-4e94-be77-28ab021314f5\") " pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:03:35.178107 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:35.178071    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9swxx"]
Jul 09 15:03:35.181141 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:03:35.181111    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fb7d2e1_0cc8_4827_a59c_59487768f9d3.slice/crio-9c493e52ae1dad4b24882c4f092fa49bb0a16ffef163ea164c61e0028254d56a WatchSource:0}: Error finding container 9c493e52ae1dad4b24882c4f092fa49bb0a16ffef163ea164c61e0028254d56a: Status 404 returned error can't find the container with id 9c493e52ae1dad4b24882c4f092fa49bb0a16ffef163ea164c61e0028254d56a
Jul 09 15:03:35.251423 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:35.251340    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7htgt\""
Jul 09 15:03:35.253362 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:35.253345    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6ltzh"
Jul 09 15:03:35.259760 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:35.259740    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:03:35.386863 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:35.386834    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6ltzh"]
Jul 09 15:03:35.390409 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:03:35.390375    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4e083b0_d118_451b_8655_30df5353d5ed.slice/crio-b316f7a2da24336815951d6feb1600f37797f392376104842be7ecbc9d4e2870 WatchSource:0}: Error finding container b316f7a2da24336815951d6feb1600f37797f392376104842be7ecbc9d4e2870: Status 404 returned error can't find the container with id b316f7a2da24336815951d6feb1600f37797f392376104842be7ecbc9d4e2870
Jul 09 15:03:35.409569 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:35.409541    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qw9p7"]
Jul 09 15:03:35.412865 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:03:35.412836    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68d94daa_3588_4e94_be77_28ab021314f5.slice/crio-2dda07dbfeb390f83d803f2b6f9267937194ee5918cc7c2dba2a2c2f669ba66c WatchSource:0}: Error finding container 2dda07dbfeb390f83d803f2b6f9267937194ee5918cc7c2dba2a2c2f669ba66c: Status 404 returned error can't find the container with id 2dda07dbfeb390f83d803f2b6f9267937194ee5918cc7c2dba2a2c2f669ba66c
Jul 09 15:03:35.562372 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:35.562271    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9swxx" event={"ID":"5fb7d2e1-0cc8-4827-a59c-59487768f9d3","Type":"ContainerStarted","Data":"9c493e52ae1dad4b24882c4f092fa49bb0a16ffef163ea164c61e0028254d56a"}
Jul 09 15:03:35.563458 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:35.563423    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qw9p7" event={"ID":"68d94daa-3588-4e94-be77-28ab021314f5","Type":"ContainerStarted","Data":"2dda07dbfeb390f83d803f2b6f9267937194ee5918cc7c2dba2a2c2f669ba66c"}
Jul 09 15:03:35.564558 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:35.564528    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6ltzh" event={"ID":"a4e083b0-d118-451b-8655-30df5353d5ed","Type":"ContainerStarted","Data":"b316f7a2da24336815951d6feb1600f37797f392376104842be7ecbc9d4e2870"}
Jul 09 15:03:38.580868 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:38.580763    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9swxx" event={"ID":"5fb7d2e1-0cc8-4827-a59c-59487768f9d3","Type":"ContainerStarted","Data":"3a74eb2fbfb0ba8ba5dce8a115019b3e828f78dfba9b6825a70569182e4a8809"}
Jul 09 15:03:38.580868 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:38.580832    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9swxx" event={"ID":"5fb7d2e1-0cc8-4827-a59c-59487768f9d3","Type":"ContainerStarted","Data":"4ba8684e7cc3f6912f5408f51439d3a93ef108198dc2da443a4ba523f4f9fb2d"}
Jul 09 15:03:38.602889 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:38.602756    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9swxx" podStartSLOduration=65.650573038 podStartE2EDuration="1m8.602732192s" totalImagesPullingTime="2.952159154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:38.602481079 +0000 UTC m=+68.935911044" watchObservedRunningTime="2026-07-09 15:03:38.602732192 +0000 UTC m=+68.936162159"
Jul 09 15:03:41.594667 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:41.594571    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qw9p7" event={"ID":"68d94daa-3588-4e94-be77-28ab021314f5","Type":"ContainerStarted","Data":"82fec31694489dd579146ebf7835be9724d07514cc20e74e4c3f5501be4487be"}
Jul 09 15:03:41.594667 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:41.594665    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:03:41.596035 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:41.596012    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6ltzh" event={"ID":"a4e083b0-d118-451b-8655-30df5353d5ed","Type":"ContainerStarted","Data":"458a284f6f55b3d1284b562f32136ec3a935c1607a1ad33ba54efd8976ea2dd0"}
Jul 09 15:03:41.617500 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:41.617455    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qw9p7" podStartSLOduration=65.719634835 podStartE2EDuration="1m11.617439397s" totalImagesPullingTime="5.897804562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:41.616269025 +0000 UTC m=+71.949698984" watchObservedRunningTime="2026-07-09 15:03:41.617439397 +0000 UTC m=+71.950869361"
Jul 09 15:03:41.635356 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:41.635309    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6ltzh" podStartSLOduration=65.710829186 podStartE2EDuration="1m11.635293181s" totalImagesPullingTime="5.924463995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:41.634428665 +0000 UTC m=+71.967858859" watchObservedRunningTime="2026-07-09 15:03:41.635293181 +0000 UTC m=+71.968723146"
Jul 09 15:03:47.523125 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:47.523096    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56b7c85567-zdl9d"
Jul 09 15:03:49.195902 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.195862    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xm86z"]
Jul 09 15:03:49.198828 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.198810    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.201907 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.201884    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nz6qf\""
Jul 09 15:03:49.201907 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.201884    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 15:03:49.202616 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.202599    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 15:03:49.202661 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.202616    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 15:03:49.203319 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.203296    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 15:03:49.203611 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.203597    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 15:03:49.203831 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.203814    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 15:03:49.288835 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.288794    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aada1234-5724-435a-a621-d686c031135f-node-exporter-textfile\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.288835 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.288832    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aada1234-5724-435a-a621-d686c031135f-node-exporter-tls\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.289062 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.288855    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aada1234-5724-435a-a621-d686c031135f-root\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.289062 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.288959    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aada1234-5724-435a-a621-d686c031135f-node-exporter-accelerators-collector-config\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.289062 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.288991    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aada1234-5724-435a-a621-d686c031135f-node-exporter-wtmp\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.289062 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.289015    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aada1234-5724-435a-a621-d686c031135f-sys\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.289062 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.289031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aada1234-5724-435a-a621-d686c031135f-metrics-client-ca\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.289257 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.289103    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aada1234-5724-435a-a621-d686c031135f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.289257 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.289152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqd6h\" (UniqueName: \"kubernetes.io/projected/aada1234-5724-435a-a621-d686c031135f-kube-api-access-nqd6h\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.389704 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.389650    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aada1234-5724-435a-a621-d686c031135f-node-exporter-accelerators-collector-config\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.389863 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.389725    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aada1234-5724-435a-a621-d686c031135f-node-exporter-wtmp\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.389863 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.389745    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aada1234-5724-435a-a621-d686c031135f-sys\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.389863 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.389763    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aada1234-5724-435a-a621-d686c031135f-metrics-client-ca\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.389863 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.389799    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aada1234-5724-435a-a621-d686c031135f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.389863 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.389821    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqd6h\" (UniqueName: \"kubernetes.io/projected/aada1234-5724-435a-a621-d686c031135f-kube-api-access-nqd6h\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.389863 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.389844    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aada1234-5724-435a-a621-d686c031135f-node-exporter-textfile\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.389863 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.389866    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aada1234-5724-435a-a621-d686c031135f-node-exporter-tls\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.390202 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.389881    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aada1234-5724-435a-a621-d686c031135f-sys\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.390202 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.389893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aada1234-5724-435a-a621-d686c031135f-root\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.390202 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.389924    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aada1234-5724-435a-a621-d686c031135f-node-exporter-wtmp\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.390202 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.390056    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aada1234-5724-435a-a621-d686c031135f-root\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.390425 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.390306    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aada1234-5724-435a-a621-d686c031135f-node-exporter-textfile\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.390425 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.390370    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aada1234-5724-435a-a621-d686c031135f-node-exporter-accelerators-collector-config\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.390425 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.390375    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aada1234-5724-435a-a621-d686c031135f-metrics-client-ca\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.392214 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.392194    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aada1234-5724-435a-a621-d686c031135f-node-exporter-tls\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.392273 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.392223    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aada1234-5724-435a-a621-d686c031135f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.402037 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.402010    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqd6h\" (UniqueName: \"kubernetes.io/projected/aada1234-5724-435a-a621-d686c031135f-kube-api-access-nqd6h\") pod \"node-exporter-xm86z\" (UID: \"aada1234-5724-435a-a621-d686c031135f\") " pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.499959 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.499850    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-58f9d88667-gshlx" podUID="4e597ee6-426e-4f2c-b0ad-e190913653b3" containerName="registry" containerID="cri-o://7c65e3dfcb2820acb89b96d1dbed2e68a05f92f66448731a6dfd2459fbd31386" gracePeriod=30
Jul 09 15:03:49.509492 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.509461    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xm86z"
Jul 09 15:03:49.567674 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:03:49.567631    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaada1234_5724_435a_a621_d686c031135f.slice/crio-8081c56a4ecd2f571bd9f86b7f7bdbfa03fa475cdc6654dd95669c5bcc6b8543 WatchSource:0}: Error finding container 8081c56a4ecd2f571bd9f86b7f7bdbfa03fa475cdc6654dd95669c5bcc6b8543: Status 404 returned error can't find the container with id 8081c56a4ecd2f571bd9f86b7f7bdbfa03fa475cdc6654dd95669c5bcc6b8543
Jul 09 15:03:49.624862 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.624831    2579 generic.go:356] "Generic (PLEG): container finished" podID="4e597ee6-426e-4f2c-b0ad-e190913653b3" containerID="7c65e3dfcb2820acb89b96d1dbed2e68a05f92f66448731a6dfd2459fbd31386" exitCode=0
Jul 09 15:03:49.625004 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.624904    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58f9d88667-gshlx" event={"ID":"4e597ee6-426e-4f2c-b0ad-e190913653b3","Type":"ContainerDied","Data":"7c65e3dfcb2820acb89b96d1dbed2e68a05f92f66448731a6dfd2459fbd31386"}
Jul 09 15:03:49.626162 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.626137    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xm86z" event={"ID":"aada1234-5724-435a-a621-d686c031135f","Type":"ContainerStarted","Data":"8081c56a4ecd2f571bd9f86b7f7bdbfa03fa475cdc6654dd95669c5bcc6b8543"}
Jul 09 15:03:49.740125 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.740093    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:49.792742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.792617    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e597ee6-426e-4f2c-b0ad-e190913653b3-image-registry-private-configuration\") pod \"4e597ee6-426e-4f2c-b0ad-e190913653b3\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") "
Jul 09 15:03:49.792742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.792671    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-bound-sa-token\") pod \"4e597ee6-426e-4f2c-b0ad-e190913653b3\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") "
Jul 09 15:03:49.792742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.792723    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqtll\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-kube-api-access-lqtll\") pod \"4e597ee6-426e-4f2c-b0ad-e190913653b3\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") "
Jul 09 15:03:49.793017 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.792815    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-certificates\") pod \"4e597ee6-426e-4f2c-b0ad-e190913653b3\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") "
Jul 09 15:03:49.793017 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.792871    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e597ee6-426e-4f2c-b0ad-e190913653b3-trusted-ca\") pod \"4e597ee6-426e-4f2c-b0ad-e190913653b3\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") "
Jul 09 15:03:49.793017 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.792898    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls\") pod \"4e597ee6-426e-4f2c-b0ad-e190913653b3\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") "
Jul 09 15:03:49.793017 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.792918    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e597ee6-426e-4f2c-b0ad-e190913653b3-installation-pull-secrets\") pod \"4e597ee6-426e-4f2c-b0ad-e190913653b3\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") "
Jul 09 15:03:49.793017 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.792949    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e597ee6-426e-4f2c-b0ad-e190913653b3-ca-trust-extracted\") pod \"4e597ee6-426e-4f2c-b0ad-e190913653b3\" (UID: \"4e597ee6-426e-4f2c-b0ad-e190913653b3\") "
Jul 09 15:03:49.793271 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.793244    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4e597ee6-426e-4f2c-b0ad-e190913653b3" (UID: "4e597ee6-426e-4f2c-b0ad-e190913653b3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:03:49.793792 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.793755    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e597ee6-426e-4f2c-b0ad-e190913653b3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4e597ee6-426e-4f2c-b0ad-e190913653b3" (UID: "4e597ee6-426e-4f2c-b0ad-e190913653b3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:03:49.795265 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.795224    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e597ee6-426e-4f2c-b0ad-e190913653b3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4e597ee6-426e-4f2c-b0ad-e190913653b3" (UID: "4e597ee6-426e-4f2c-b0ad-e190913653b3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:03:49.795371 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.795292    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-kube-api-access-lqtll" (OuterVolumeSpecName: "kube-api-access-lqtll") pod "4e597ee6-426e-4f2c-b0ad-e190913653b3" (UID: "4e597ee6-426e-4f2c-b0ad-e190913653b3"). InnerVolumeSpecName "kube-api-access-lqtll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:03:49.795371 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.795334    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4e597ee6-426e-4f2c-b0ad-e190913653b3" (UID: "4e597ee6-426e-4f2c-b0ad-e190913653b3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:03:49.795493 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.795439    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e597ee6-426e-4f2c-b0ad-e190913653b3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4e597ee6-426e-4f2c-b0ad-e190913653b3" (UID: "4e597ee6-426e-4f2c-b0ad-e190913653b3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:03:49.795544 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.795495    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4e597ee6-426e-4f2c-b0ad-e190913653b3" (UID: "4e597ee6-426e-4f2c-b0ad-e190913653b3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:03:49.801970 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.801944    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e597ee6-426e-4f2c-b0ad-e190913653b3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4e597ee6-426e-4f2c-b0ad-e190913653b3" (UID: "4e597ee6-426e-4f2c-b0ad-e190913653b3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:03:49.894166 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.894114    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4e597ee6-426e-4f2c-b0ad-e190913653b3-image-registry-private-configuration\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:03:49.894166 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.894159    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-bound-sa-token\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:03:49.894166 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.894170    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqtll\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-kube-api-access-lqtll\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:03:49.894166 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.894179    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-certificates\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:03:49.894387 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.894188    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4e597ee6-426e-4f2c-b0ad-e190913653b3-trusted-ca\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:03:49.894387 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.894196    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4e597ee6-426e-4f2c-b0ad-e190913653b3-registry-tls\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:03:49.894387 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.894204    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4e597ee6-426e-4f2c-b0ad-e190913653b3-installation-pull-secrets\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:03:49.894387 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:49.894211    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4e597ee6-426e-4f2c-b0ad-e190913653b3-ca-trust-extracted\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:03:50.632093 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:50.631998    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58f9d88667-gshlx" event={"ID":"4e597ee6-426e-4f2c-b0ad-e190913653b3","Type":"ContainerDied","Data":"c6e9eeb0e65feb8bd634cf0a62cfb98d327a01071db35408e771ceccade70d4c"}
Jul 09 15:03:50.632093 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:50.632027    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58f9d88667-gshlx"
Jul 09 15:03:50.632093 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:50.632061    2579 scope.go:117] "RemoveContainer" containerID="7c65e3dfcb2820acb89b96d1dbed2e68a05f92f66448731a6dfd2459fbd31386"
Jul 09 15:03:50.652135 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:50.652093    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-58f9d88667-gshlx"]
Jul 09 15:03:50.656476 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:50.656446    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-58f9d88667-gshlx"]
Jul 09 15:03:51.637359 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:51.637326    2579 generic.go:356] "Generic (PLEG): container finished" podID="aada1234-5724-435a-a621-d686c031135f" containerID="76fa1533cabe0b107f208f37af6dbb8c582a1d6c5c34ccd2242ec443550a3a89" exitCode=0
Jul 09 15:03:51.637806 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:51.637413    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xm86z" event={"ID":"aada1234-5724-435a-a621-d686c031135f","Type":"ContainerDied","Data":"76fa1533cabe0b107f208f37af6dbb8c582a1d6c5c34ccd2242ec443550a3a89"}
Jul 09 15:03:52.235223 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:52.235180    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e597ee6-426e-4f2c-b0ad-e190913653b3" path="/var/lib/kubelet/pods/4e597ee6-426e-4f2c-b0ad-e190913653b3/volumes"
Jul 09 15:03:52.645085 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:52.644978    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xm86z" event={"ID":"aada1234-5724-435a-a621-d686c031135f","Type":"ContainerStarted","Data":"30ad2d2bd32a3cf01173660627c5ec8849a0163ee7e1db3c622259ba381eb1d2"}
Jul 09 15:03:52.645085 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:52.645047    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xm86z" event={"ID":"aada1234-5724-435a-a621-d686c031135f","Type":"ContainerStarted","Data":"f634e5fccc00e45d3d511426ef4c98b8f0dfc001335280a64ab90ed64468ae51"}
Jul 09 15:03:52.668866 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:03:52.668808    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xm86z" podStartSLOduration=2.082674678 podStartE2EDuration="3.668792182s" totalImagesPullingTime="1.516725904s" totalInitContainerRuntime="69.3916ms" isStatefulPod=true podCreationTimestamp="2026-07-09 15:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:03:52.667748918 +0000 UTC m=+83.001178882" watchObservedRunningTime="2026-07-09 15:03:52.668792182 +0000 UTC m=+83.002222146"
Jul 09 15:04:12.602437 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:04:12.602403    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qw9p7"
Jul 09 15:04:28.148657 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:04:28.148620    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q86vm_0817217b-29ab-4c7d-8f05-874d0fc2b557/serve-healthcheck-canary/0.log"
Jul 09 15:07:30.056265 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:07:30.056233    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:07:30.059867 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:07:30.059838    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:07:30.065787 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:07:30.065762    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 15:09:23.253721 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.253666    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"]
Jul 09 15:09:23.254252 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.254043    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e597ee6-426e-4f2c-b0ad-e190913653b3" containerName="registry"
Jul 09 15:09:23.254252 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.254063    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e597ee6-426e-4f2c-b0ad-e190913653b3" containerName="registry"
Jul 09 15:09:23.254252 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.254139    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e597ee6-426e-4f2c-b0ad-e190913653b3" containerName="registry"
Jul 09 15:09:23.255888 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.255867    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.258894 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.258873    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 09 15:09:23.259169 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.259154    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 09 15:09:23.259220 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.259201    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 09 15:09:23.259492 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.259472    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5hbzl\""
Jul 09 15:09:23.259492 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.259473    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 09 15:09:23.278893 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.278864    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"]
Jul 09 15:09:23.346384 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.346344    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2917027-3953-4933-b7a4-7b5f7c45d6ef-tmp\") pod \"opendatahub-operator-controller-manager-66b8fbd475-8gj8v\" (UID: \"f2917027-3953-4933-b7a4-7b5f7c45d6ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.346384 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.346387    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2917027-3953-4933-b7a4-7b5f7c45d6ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-66b8fbd475-8gj8v\" (UID: \"f2917027-3953-4933-b7a4-7b5f7c45d6ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.346655 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.346412    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2917027-3953-4933-b7a4-7b5f7c45d6ef-manifests\") pod \"opendatahub-operator-controller-manager-66b8fbd475-8gj8v\" (UID: \"f2917027-3953-4933-b7a4-7b5f7c45d6ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.346655 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.346530    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2917027-3953-4933-b7a4-7b5f7c45d6ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66b8fbd475-8gj8v\" (UID: \"f2917027-3953-4933-b7a4-7b5f7c45d6ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.346655 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.346570    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqzfj\" (UniqueName: \"kubernetes.io/projected/f2917027-3953-4933-b7a4-7b5f7c45d6ef-kube-api-access-xqzfj\") pod \"opendatahub-operator-controller-manager-66b8fbd475-8gj8v\" (UID: \"f2917027-3953-4933-b7a4-7b5f7c45d6ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.447988 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.447951    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2917027-3953-4933-b7a4-7b5f7c45d6ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66b8fbd475-8gj8v\" (UID: \"f2917027-3953-4933-b7a4-7b5f7c45d6ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.448132 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.448084    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqzfj\" (UniqueName: \"kubernetes.io/projected/f2917027-3953-4933-b7a4-7b5f7c45d6ef-kube-api-access-xqzfj\") pod \"opendatahub-operator-controller-manager-66b8fbd475-8gj8v\" (UID: \"f2917027-3953-4933-b7a4-7b5f7c45d6ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.448132 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.448123    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2917027-3953-4933-b7a4-7b5f7c45d6ef-tmp\") pod \"opendatahub-operator-controller-manager-66b8fbd475-8gj8v\" (UID: \"f2917027-3953-4933-b7a4-7b5f7c45d6ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.448246 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.448170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2917027-3953-4933-b7a4-7b5f7c45d6ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-66b8fbd475-8gj8v\" (UID: \"f2917027-3953-4933-b7a4-7b5f7c45d6ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.448246 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.448213    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2917027-3953-4933-b7a4-7b5f7c45d6ef-manifests\") pod \"opendatahub-operator-controller-manager-66b8fbd475-8gj8v\" (UID: \"f2917027-3953-4933-b7a4-7b5f7c45d6ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.448568 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.448549    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2917027-3953-4933-b7a4-7b5f7c45d6ef-manifests\") pod \"opendatahub-operator-controller-manager-66b8fbd475-8gj8v\" (UID: \"f2917027-3953-4933-b7a4-7b5f7c45d6ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.450222 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.450206    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2917027-3953-4933-b7a4-7b5f7c45d6ef-tmp\") pod \"opendatahub-operator-controller-manager-66b8fbd475-8gj8v\" (UID: \"f2917027-3953-4933-b7a4-7b5f7c45d6ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.450481 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.450465    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2917027-3953-4933-b7a4-7b5f7c45d6ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-66b8fbd475-8gj8v\" (UID: \"f2917027-3953-4933-b7a4-7b5f7c45d6ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.450540 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.450518    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2917027-3953-4933-b7a4-7b5f7c45d6ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66b8fbd475-8gj8v\" (UID: \"f2917027-3953-4933-b7a4-7b5f7c45d6ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.458800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.458772    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqzfj\" (UniqueName: \"kubernetes.io/projected/f2917027-3953-4933-b7a4-7b5f7c45d6ef-kube-api-access-xqzfj\") pod \"opendatahub-operator-controller-manager-66b8fbd475-8gj8v\" (UID: \"f2917027-3953-4933-b7a4-7b5f7c45d6ef\") " pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.566047 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.565954    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:23.707569 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.707543    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"]
Jul 09 15:09:23.710074 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:09:23.710050    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2917027_3953_4933_b7a4_7b5f7c45d6ef.slice/crio-973530f1fca1269af8279fc41a571b47b367745ad9a5c657b6acf31038506a10 WatchSource:0}: Error finding container 973530f1fca1269af8279fc41a571b47b367745ad9a5c657b6acf31038506a10: Status 404 returned error can't find the container with id 973530f1fca1269af8279fc41a571b47b367745ad9a5c657b6acf31038506a10
Jul 09 15:09:23.711801 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:23.711783    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 15:09:24.702373 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:24.702342    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v" event={"ID":"f2917027-3953-4933-b7a4-7b5f7c45d6ef","Type":"ContainerStarted","Data":"973530f1fca1269af8279fc41a571b47b367745ad9a5c657b6acf31038506a10"}
Jul 09 15:09:27.717221 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:27.717185    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v" event={"ID":"f2917027-3953-4933-b7a4-7b5f7c45d6ef","Type":"ContainerStarted","Data":"20cf16a42f58729f1b013cfdf0fe9b24deca3bab4a4979e5cf80667c7a7989c5"}
Jul 09 15:09:28.724189 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:28.724156    2579 generic.go:356] "Generic (PLEG): container finished" podID="f2917027-3953-4933-b7a4-7b5f7c45d6ef" containerID="20cf16a42f58729f1b013cfdf0fe9b24deca3bab4a4979e5cf80667c7a7989c5" exitCode=0
Jul 09 15:09:28.724647 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:28.724188    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v" event={"ID":"f2917027-3953-4933-b7a4-7b5f7c45d6ef","Type":"ContainerDied","Data":"20cf16a42f58729f1b013cfdf0fe9b24deca3bab4a4979e5cf80667c7a7989c5"}
Jul 09 15:09:29.729850 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:29.729818    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v" event={"ID":"f2917027-3953-4933-b7a4-7b5f7c45d6ef","Type":"ContainerStarted","Data":"23cba587109d86f0785c39777399377f2e0239226f4ba85f2a13cb5ad06d68b4"}
Jul 09 15:09:29.730242 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:29.729938    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:29.780922 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:29.780876    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v" podStartSLOduration=1.248104823 podStartE2EDuration="6.780857697s" totalImagesPullingTime="4.445194865s" totalInitContainerRuntime="1.087558009s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:09:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:09:29.778416728 +0000 UTC m=+420.111846691" watchObservedRunningTime="2026-07-09 15:09:29.780857697 +0000 UTC m=+420.114287661"
Jul 09 15:09:40.735924 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:40.735894    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-66b8fbd475-8gj8v"
Jul 09 15:09:43.227772 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.227733    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx"]
Jul 09 15:09:43.239100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.239068    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx"
Jul 09 15:09:43.243504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.243480    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 09 15:09:43.244226 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.244201    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cn25d\""
Jul 09 15:09:43.244744 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.244387    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 09 15:09:43.244897 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.244877    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 09 15:09:43.245122 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.245109    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 09 15:09:43.246405 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.246386    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx"]
Jul 09 15:09:43.391938 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.391898    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbjf9\" (UniqueName: \"kubernetes.io/projected/8f8da16e-38d9-48b8-87f0-4925aa46ed7d-kube-api-access-wbjf9\") pod \"kube-auth-proxy-b8bb9d8f7-s5tjx\" (UID: \"8f8da16e-38d9-48b8-87f0-4925aa46ed7d\") " pod="openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx"
Jul 09 15:09:43.392120 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.391946    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f8da16e-38d9-48b8-87f0-4925aa46ed7d-tmp\") pod \"kube-auth-proxy-b8bb9d8f7-s5tjx\" (UID: \"8f8da16e-38d9-48b8-87f0-4925aa46ed7d\") " pod="openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx"
Jul 09 15:09:43.392120 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.392021    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f8da16e-38d9-48b8-87f0-4925aa46ed7d-tls-certs\") pod \"kube-auth-proxy-b8bb9d8f7-s5tjx\" (UID: \"8f8da16e-38d9-48b8-87f0-4925aa46ed7d\") " pod="openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx"
Jul 09 15:09:43.492475 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.492379    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f8da16e-38d9-48b8-87f0-4925aa46ed7d-tls-certs\") pod \"kube-auth-proxy-b8bb9d8f7-s5tjx\" (UID: \"8f8da16e-38d9-48b8-87f0-4925aa46ed7d\") " pod="openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx"
Jul 09 15:09:43.492475 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.492436    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbjf9\" (UniqueName: \"kubernetes.io/projected/8f8da16e-38d9-48b8-87f0-4925aa46ed7d-kube-api-access-wbjf9\") pod \"kube-auth-proxy-b8bb9d8f7-s5tjx\" (UID: \"8f8da16e-38d9-48b8-87f0-4925aa46ed7d\") " pod="openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx"
Jul 09 15:09:43.492475 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.492470    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f8da16e-38d9-48b8-87f0-4925aa46ed7d-tmp\") pod \"kube-auth-proxy-b8bb9d8f7-s5tjx\" (UID: \"8f8da16e-38d9-48b8-87f0-4925aa46ed7d\") " pod="openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx"
Jul 09 15:09:43.494775 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.494744    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f8da16e-38d9-48b8-87f0-4925aa46ed7d-tmp\") pod \"kube-auth-proxy-b8bb9d8f7-s5tjx\" (UID: \"8f8da16e-38d9-48b8-87f0-4925aa46ed7d\") " pod="openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx"
Jul 09 15:09:43.495020 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.494992    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f8da16e-38d9-48b8-87f0-4925aa46ed7d-tls-certs\") pod \"kube-auth-proxy-b8bb9d8f7-s5tjx\" (UID: \"8f8da16e-38d9-48b8-87f0-4925aa46ed7d\") " pod="openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx"
Jul 09 15:09:43.495464 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.495446    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qdln4"]
Jul 09 15:09:43.497489 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.497472    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qdln4"
Jul 09 15:09:43.499881 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.499850    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-lsfd2\""
Jul 09 15:09:43.500249 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.500211    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 09 15:09:43.507844 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.507825    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbjf9\" (UniqueName: \"kubernetes.io/projected/8f8da16e-38d9-48b8-87f0-4925aa46ed7d-kube-api-access-wbjf9\") pod \"kube-auth-proxy-b8bb9d8f7-s5tjx\" (UID: \"8f8da16e-38d9-48b8-87f0-4925aa46ed7d\") " pod="openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx"
Jul 09 15:09:43.518648 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.518622    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qdln4"]
Jul 09 15:09:43.548804 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.548776    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx"
Jul 09 15:09:43.593486 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.593411    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8s29\" (UniqueName: \"kubernetes.io/projected/5407f877-73d9-49b5-856b-a880d09f3963-kube-api-access-t8s29\") pod \"odh-model-controller-7998c67dc4-qdln4\" (UID: \"5407f877-73d9-49b5-856b-a880d09f3963\") " pod="opendatahub/odh-model-controller-7998c67dc4-qdln4"
Jul 09 15:09:43.593649 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.593517    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5407f877-73d9-49b5-856b-a880d09f3963-cert\") pod \"odh-model-controller-7998c67dc4-qdln4\" (UID: \"5407f877-73d9-49b5-856b-a880d09f3963\") " pod="opendatahub/odh-model-controller-7998c67dc4-qdln4"
Jul 09 15:09:43.694193 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.694152    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8s29\" (UniqueName: \"kubernetes.io/projected/5407f877-73d9-49b5-856b-a880d09f3963-kube-api-access-t8s29\") pod \"odh-model-controller-7998c67dc4-qdln4\" (UID: \"5407f877-73d9-49b5-856b-a880d09f3963\") " pod="opendatahub/odh-model-controller-7998c67dc4-qdln4"
Jul 09 15:09:43.694401 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.694223    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5407f877-73d9-49b5-856b-a880d09f3963-cert\") pod \"odh-model-controller-7998c67dc4-qdln4\" (UID: \"5407f877-73d9-49b5-856b-a880d09f3963\") " pod="opendatahub/odh-model-controller-7998c67dc4-qdln4"
Jul 09 15:09:43.694401 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:09:43.694318    2579 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 09 15:09:43.694401 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:09:43.694378    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5407f877-73d9-49b5-856b-a880d09f3963-cert podName:5407f877-73d9-49b5-856b-a880d09f3963 nodeName:}" failed. No retries permitted until 2026-07-09 15:09:44.194360926 +0000 UTC m=+434.527790875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5407f877-73d9-49b5-856b-a880d09f3963-cert") pod "odh-model-controller-7998c67dc4-qdln4" (UID: "5407f877-73d9-49b5-856b-a880d09f3963") : secret "odh-model-controller-webhook-cert" not found
Jul 09 15:09:43.700026 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.700000    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx"]
Jul 09 15:09:43.703250 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:09:43.703227    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f8da16e_38d9_48b8_87f0_4925aa46ed7d.slice/crio-cecebc8f66a80926b5c703f55a9951c5bdf7be5cedc9936f55e7912e40e82f1b WatchSource:0}: Error finding container cecebc8f66a80926b5c703f55a9951c5bdf7be5cedc9936f55e7912e40e82f1b: Status 404 returned error can't find the container with id cecebc8f66a80926b5c703f55a9951c5bdf7be5cedc9936f55e7912e40e82f1b
Jul 09 15:09:43.723144 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.723118    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8s29\" (UniqueName: \"kubernetes.io/projected/5407f877-73d9-49b5-856b-a880d09f3963-kube-api-access-t8s29\") pod \"odh-model-controller-7998c67dc4-qdln4\" (UID: \"5407f877-73d9-49b5-856b-a880d09f3963\") " pod="opendatahub/odh-model-controller-7998c67dc4-qdln4"
Jul 09 15:09:43.773638 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:43.773553    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx" event={"ID":"8f8da16e-38d9-48b8-87f0-4925aa46ed7d","Type":"ContainerStarted","Data":"cecebc8f66a80926b5c703f55a9951c5bdf7be5cedc9936f55e7912e40e82f1b"}
Jul 09 15:09:44.198243 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.198209    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5407f877-73d9-49b5-856b-a880d09f3963-cert\") pod \"odh-model-controller-7998c67dc4-qdln4\" (UID: \"5407f877-73d9-49b5-856b-a880d09f3963\") " pod="opendatahub/odh-model-controller-7998c67dc4-qdln4"
Jul 09 15:09:44.200517 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.200496    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5407f877-73d9-49b5-856b-a880d09f3963-cert\") pod \"odh-model-controller-7998c67dc4-qdln4\" (UID: \"5407f877-73d9-49b5-856b-a880d09f3963\") " pod="opendatahub/odh-model-controller-7998c67dc4-qdln4"
Jul 09 15:09:44.418585 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.418545    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qdln4"
Jul 09 15:09:44.546930 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.546907    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qdln4"]
Jul 09 15:09:44.549635 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:09:44.549603    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5407f877_73d9_49b5_856b_a880d09f3963.slice/crio-ccf45e1ad901ecb34388d6d0488d419416bac3c5f945348faabd28576ffed600 WatchSource:0}: Error finding container ccf45e1ad901ecb34388d6d0488d419416bac3c5f945348faabd28576ffed600: Status 404 returned error can't find the container with id ccf45e1ad901ecb34388d6d0488d419416bac3c5f945348faabd28576ffed600
Jul 09 15:09:44.778098 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.778013    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qdln4" event={"ID":"5407f877-73d9-49b5-856b-a880d09f3963","Type":"ContainerStarted","Data":"ccf45e1ad901ecb34388d6d0488d419416bac3c5f945348faabd28576ffed600"}
Jul 09 15:09:44.813560 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.813528    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"]
Jul 09 15:09:44.816301 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.816278    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:44.819918 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.819895    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vs9h9\""
Jul 09 15:09:44.820061 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.819896    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 09 15:09:44.820061 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.819939    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 09 15:09:44.820155 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.820075    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 09 15:09:44.820546 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.820533    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 09 15:09:44.832107 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.832079    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"]
Jul 09 15:09:44.904345 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.904304    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c89881d-86da-45d9-8871-0bf08d9291bf-local-certs\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:44.904345 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.904343    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzp86\" (UniqueName: \"kubernetes.io/projected/3c89881d-86da-45d9-8871-0bf08d9291bf-kube-api-access-pzp86\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:44.904604 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.904367    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:44.904604 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.904470    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:44.904604 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.904507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-token\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:44.904604 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.904540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:44.904604 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:44.904571    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-cacerts\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.005297 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.005255    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c89881d-86da-45d9-8871-0bf08d9291bf-local-certs\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.005297 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.005295    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzp86\" (UniqueName: \"kubernetes.io/projected/3c89881d-86da-45d9-8871-0bf08d9291bf-kube-api-access-pzp86\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.005534 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.005320    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.005534 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.005355    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.005534 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.005370    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-token\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.005534 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.005390    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.005534 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.005411    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-cacerts\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.006204 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.006177    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.007867 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.007843    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-cacerts\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.007982 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.007869    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.007982 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.007945    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c89881d-86da-45d9-8871-0bf08d9291bf-local-certs\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.008153 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.008132    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.018351 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.018326    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-token\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.019152 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.019122    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzp86\" (UniqueName: \"kubernetes.io/projected/3c89881d-86da-45d9-8871-0bf08d9291bf-kube-api-access-pzp86\") pod \"istiod-openshift-gateway-7574459656-nfxlg\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.125699 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.125605    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:45.251648 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.251616    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"]
Jul 09 15:09:45.254371 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:09:45.254347    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c89881d_86da_45d9_8871_0bf08d9291bf.slice/crio-b40e015fdce0ce515b57396bcaef919a3a38aa24dce0bb915c8787a3d7fef72b WatchSource:0}: Error finding container b40e015fdce0ce515b57396bcaef919a3a38aa24dce0bb915c8787a3d7fef72b: Status 404 returned error can't find the container with id b40e015fdce0ce515b57396bcaef919a3a38aa24dce0bb915c8787a3d7fef72b
Jul 09 15:09:45.783521 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:45.783479    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg" event={"ID":"3c89881d-86da-45d9-8871-0bf08d9291bf","Type":"ContainerStarted","Data":"b40e015fdce0ce515b57396bcaef919a3a38aa24dce0bb915c8787a3d7fef72b"}
Jul 09 15:09:48.797577 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:48.797542    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx" event={"ID":"8f8da16e-38d9-48b8-87f0-4925aa46ed7d","Type":"ContainerStarted","Data":"9cd230ecff6349c1521fb5430f58de9f988ce7b33b6847e6017bfc3baf7dcde0"}
Jul 09 15:09:48.799171 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:48.799142    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qdln4" event={"ID":"5407f877-73d9-49b5-856b-a880d09f3963","Type":"ContainerStarted","Data":"1e89162b00d2908b099beba0703eb28434ae3f737d823bb08017bbf376f089e6"}
Jul 09 15:09:48.799318 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:48.799273    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-qdln4"
Jul 09 15:09:48.817814 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:48.817752    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-b8bb9d8f7-s5tjx" podStartSLOduration=0.868690872 podStartE2EDuration="5.817730586s" totalImagesPullingTime="4.949039714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:09:48.81715025 +0000 UTC m=+439.150580215" watchObservedRunningTime="2026-07-09 15:09:48.817730586 +0000 UTC m=+439.151160553"
Jul 09 15:09:48.838501 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:48.838453    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-qdln4" podStartSLOduration=1.7357936299999999 podStartE2EDuration="5.838436699s" totalImagesPullingTime="4.102643069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:09:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:09:48.837225354 +0000 UTC m=+439.170655319" watchObservedRunningTime="2026-07-09 15:09:48.838436699 +0000 UTC m=+439.171866663"
Jul 09 15:09:49.609536 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:49.609500    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 15:09:49.609649 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:49.609563    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 09 15:09:49.804334 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:49.804291    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg" event={"ID":"3c89881d-86da-45d9-8871-0bf08d9291bf","Type":"ContainerStarted","Data":"88370013afad15a195735eb4f2e06f4c7dfadba7ced1aeae11070257603d4e8a"}
Jul 09 15:09:49.804747 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:49.804599    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:49.832601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:49.832558    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg" podStartSLOduration=1.479602127 podStartE2EDuration="5.832541727s" totalImagesPullingTime="4.3529396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:09:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:09:49.831840043 +0000 UTC m=+440.165270009" watchObservedRunningTime="2026-07-09 15:09:49.832541727 +0000 UTC m=+440.165971688"
Jul 09 15:09:50.810362 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:50.810323    2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-nfxlg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 09 15:09:50.810756 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:50.810377    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg" podUID="3c89881d-86da-45d9-8871-0bf08d9291bf" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:09:53.809504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:53.809429    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:54.520864 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:54.520827    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44"]
Jul 09 15:09:54.523671 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:54.523648    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44"
Jul 09 15:09:54.527572 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:54.527554    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 09 15:09:54.530858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:54.530839    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-c7xg4\""
Jul 09 15:09:54.545655 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:54.545622    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44"]
Jul 09 15:09:54.583980 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:54.583953    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng2ch\" (UniqueName: \"kubernetes.io/projected/1c395aa0-45df-428a-aace-f93e3325e9f8-kube-api-access-ng2ch\") pod \"llmisvc-controller-manager-777c6bd7d5-jdx44\" (UID: \"1c395aa0-45df-428a-aace-f93e3325e9f8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44"
Jul 09 15:09:54.584118 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:54.583996    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c395aa0-45df-428a-aace-f93e3325e9f8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jdx44\" (UID: \"1c395aa0-45df-428a-aace-f93e3325e9f8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44"
Jul 09 15:09:54.684988 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:54.684954    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c395aa0-45df-428a-aace-f93e3325e9f8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jdx44\" (UID: \"1c395aa0-45df-428a-aace-f93e3325e9f8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44"
Jul 09 15:09:54.685161 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:54.685009    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng2ch\" (UniqueName: \"kubernetes.io/projected/1c395aa0-45df-428a-aace-f93e3325e9f8-kube-api-access-ng2ch\") pod \"llmisvc-controller-manager-777c6bd7d5-jdx44\" (UID: \"1c395aa0-45df-428a-aace-f93e3325e9f8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44"
Jul 09 15:09:54.687260 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:54.687236    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c395aa0-45df-428a-aace-f93e3325e9f8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jdx44\" (UID: \"1c395aa0-45df-428a-aace-f93e3325e9f8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44"
Jul 09 15:09:54.700870 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:54.700845    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng2ch\" (UniqueName: \"kubernetes.io/projected/1c395aa0-45df-428a-aace-f93e3325e9f8-kube-api-access-ng2ch\") pod \"llmisvc-controller-manager-777c6bd7d5-jdx44\" (UID: \"1c395aa0-45df-428a-aace-f93e3325e9f8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44"
Jul 09 15:09:54.833579 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:54.833482    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44"
Jul 09 15:09:54.994186 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:54.994152    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44"]
Jul 09 15:09:54.997317 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:09:54.997288    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1c395aa0_45df_428a_aace_f93e3325e9f8.slice/crio-b2db47a02ac09eedc29d050a1ba0434991aed90fa92ff5924847cb3de2a3d443 WatchSource:0}: Error finding container b2db47a02ac09eedc29d050a1ba0434991aed90fa92ff5924847cb3de2a3d443: Status 404 returned error can't find the container with id b2db47a02ac09eedc29d050a1ba0434991aed90fa92ff5924847cb3de2a3d443
Jul 09 15:09:55.827068 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:55.827037    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44" event={"ID":"1c395aa0-45df-428a-aace-f93e3325e9f8","Type":"ContainerStarted","Data":"b2db47a02ac09eedc29d050a1ba0434991aed90fa92ff5924847cb3de2a3d443"}
Jul 09 15:09:58.838795 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:58.838759    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44" event={"ID":"1c395aa0-45df-428a-aace-f93e3325e9f8","Type":"ContainerStarted","Data":"999d7f89f25e4b1489a41b28703ac77088e7cfbd52d0f4f883813d18b8fae1ec"}
Jul 09 15:09:58.839206 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:58.838890    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44"
Jul 09 15:09:58.874228 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:58.874172    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44" podStartSLOduration=1.510327056 podStartE2EDuration="4.874154694s" totalImagesPullingTime="3.363827638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:09:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:09:58.872705644 +0000 UTC m=+449.206135599" watchObservedRunningTime="2026-07-09 15:09:58.874154694 +0000 UTC m=+449.207584657"
Jul 09 15:09:58.916521 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:58.916493    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"]
Jul 09 15:09:58.916721 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:58.916663    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg" podUID="3c89881d-86da-45d9-8871-0bf08d9291bf" containerName="discovery" containerID="cri-o://88370013afad15a195735eb4f2e06f4c7dfadba7ced1aeae11070257603d4e8a" gracePeriod=30
Jul 09 15:09:59.151077 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.151054    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:59.216351 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.216323    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-token\") pod \"3c89881d-86da-45d9-8871-0bf08d9291bf\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") "
Jul 09 15:09:59.216508 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.216361    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-csr-ca-configmap\") pod \"3c89881d-86da-45d9-8871-0bf08d9291bf\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") "
Jul 09 15:09:59.216508 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.216380    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-csr-dns-cert\") pod \"3c89881d-86da-45d9-8871-0bf08d9291bf\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") "
Jul 09 15:09:59.216508 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.216414    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-cacerts\") pod \"3c89881d-86da-45d9-8871-0bf08d9291bf\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") "
Jul 09 15:09:59.216508 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.216441    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzp86\" (UniqueName: \"kubernetes.io/projected/3c89881d-86da-45d9-8871-0bf08d9291bf-kube-api-access-pzp86\") pod \"3c89881d-86da-45d9-8871-0bf08d9291bf\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") "
Jul 09 15:09:59.216508 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.216485    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c89881d-86da-45d9-8871-0bf08d9291bf-local-certs\") pod \"3c89881d-86da-45d9-8871-0bf08d9291bf\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") "
Jul 09 15:09:59.216508 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.216508    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-kubeconfig\") pod \"3c89881d-86da-45d9-8871-0bf08d9291bf\" (UID: \"3c89881d-86da-45d9-8871-0bf08d9291bf\") "
Jul 09 15:09:59.216828 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.216767    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "3c89881d-86da-45d9-8871-0bf08d9291bf" (UID: "3c89881d-86da-45d9-8871-0bf08d9291bf"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:09:59.218860 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.218823    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c89881d-86da-45d9-8871-0bf08d9291bf-local-certs" (OuterVolumeSpecName: "local-certs") pod "3c89881d-86da-45d9-8871-0bf08d9291bf" (UID: "3c89881d-86da-45d9-8871-0bf08d9291bf"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:09:59.219158 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.219115    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "3c89881d-86da-45d9-8871-0bf08d9291bf" (UID: "3c89881d-86da-45d9-8871-0bf08d9291bf"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:09:59.219158 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.219126    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-cacerts" (OuterVolumeSpecName: "cacerts") pod "3c89881d-86da-45d9-8871-0bf08d9291bf" (UID: "3c89881d-86da-45d9-8871-0bf08d9291bf"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:09:59.219158 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.219122    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c89881d-86da-45d9-8871-0bf08d9291bf" (UID: "3c89881d-86da-45d9-8871-0bf08d9291bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:09:59.219158 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.219148    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c89881d-86da-45d9-8871-0bf08d9291bf-kube-api-access-pzp86" (OuterVolumeSpecName: "kube-api-access-pzp86") pod "3c89881d-86da-45d9-8871-0bf08d9291bf" (UID: "3c89881d-86da-45d9-8871-0bf08d9291bf"). InnerVolumeSpecName "kube-api-access-pzp86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:09:59.219375 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.219150    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "3c89881d-86da-45d9-8871-0bf08d9291bf" (UID: "3c89881d-86da-45d9-8871-0bf08d9291bf"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:09:59.317327 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.317291    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-token\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:09:59.317327 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.317324    2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-csr-ca-configmap\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:09:59.317535 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.317339    2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-csr-dns-cert\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:09:59.317535 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.317351    2579 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-cacerts\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:09:59.317535 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.317362    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzp86\" (UniqueName: \"kubernetes.io/projected/3c89881d-86da-45d9-8871-0bf08d9291bf-kube-api-access-pzp86\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:09:59.317535 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.317373    2579 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c89881d-86da-45d9-8871-0bf08d9291bf-local-certs\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:09:59.317535 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.317384    2579 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3c89881d-86da-45d9-8871-0bf08d9291bf-istio-kubeconfig\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:09:59.807101 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.807065    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-qdln4"
Jul 09 15:09:59.844249 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.844219    2579 generic.go:356] "Generic (PLEG): container finished" podID="3c89881d-86da-45d9-8871-0bf08d9291bf" containerID="88370013afad15a195735eb4f2e06f4c7dfadba7ced1aeae11070257603d4e8a" exitCode=0
Jul 09 15:09:59.844739 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.844265    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"
Jul 09 15:09:59.844739 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.844311    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg" event={"ID":"3c89881d-86da-45d9-8871-0bf08d9291bf","Type":"ContainerDied","Data":"88370013afad15a195735eb4f2e06f4c7dfadba7ced1aeae11070257603d4e8a"}
Jul 09 15:09:59.844739 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.844368    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg" event={"ID":"3c89881d-86da-45d9-8871-0bf08d9291bf","Type":"ContainerDied","Data":"b40e015fdce0ce515b57396bcaef919a3a38aa24dce0bb915c8787a3d7fef72b"}
Jul 09 15:09:59.844739 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.844398    2579 scope.go:117] "RemoveContainer" containerID="88370013afad15a195735eb4f2e06f4c7dfadba7ced1aeae11070257603d4e8a"
Jul 09 15:09:59.853163 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.853142    2579 scope.go:117] "RemoveContainer" containerID="88370013afad15a195735eb4f2e06f4c7dfadba7ced1aeae11070257603d4e8a"
Jul 09 15:09:59.853425 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:09:59.853407    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88370013afad15a195735eb4f2e06f4c7dfadba7ced1aeae11070257603d4e8a\": container with ID starting with 88370013afad15a195735eb4f2e06f4c7dfadba7ced1aeae11070257603d4e8a not found: ID does not exist" containerID="88370013afad15a195735eb4f2e06f4c7dfadba7ced1aeae11070257603d4e8a"
Jul 09 15:09:59.853491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.853433    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88370013afad15a195735eb4f2e06f4c7dfadba7ced1aeae11070257603d4e8a"} err="failed to get container status \"88370013afad15a195735eb4f2e06f4c7dfadba7ced1aeae11070257603d4e8a\": rpc error: code = NotFound desc = could not find container \"88370013afad15a195735eb4f2e06f4c7dfadba7ced1aeae11070257603d4e8a\": container with ID starting with 88370013afad15a195735eb4f2e06f4c7dfadba7ced1aeae11070257603d4e8a not found: ID does not exist"
Jul 09 15:09:59.885142 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.885109    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"]
Jul 09 15:09:59.887258 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:09:59.887232    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nfxlg"]
Jul 09 15:10:00.234366 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:00.234336    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c89881d-86da-45d9-8871-0bf08d9291bf" path="/var/lib/kubelet/pods/3c89881d-86da-45d9-8871-0bf08d9291bf/volumes"
Jul 09 15:10:29.846631 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:29.846599    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jdx44"
Jul 09 15:10:44.341345 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.341313    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7fqc8"]
Jul 09 15:10:44.341831 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.341560    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c89881d-86da-45d9-8871-0bf08d9291bf" containerName="discovery"
Jul 09 15:10:44.341831 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.341570    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c89881d-86da-45d9-8871-0bf08d9291bf" containerName="discovery"
Jul 09 15:10:44.341831 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.341630    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c89881d-86da-45d9-8871-0bf08d9291bf" containerName="discovery"
Jul 09 15:10:44.343976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.343961    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7fqc8"
Jul 09 15:10:44.346781 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.346756    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 15:10:44.347841 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.347817    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 15:10:44.347957 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.347820    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-64zgc\""
Jul 09 15:10:44.356921 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.356890    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7fqc8"]
Jul 09 15:10:44.462348 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.462314    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54nvd\" (UniqueName: \"kubernetes.io/projected/5fbdf502-f38a-41d7-9c02-a3d408c0e420-kube-api-access-54nvd\") pod \"kuadrant-operator-catalog-7fqc8\" (UID: \"5fbdf502-f38a-41d7-9c02-a3d408c0e420\") " pod="kuadrant-system/kuadrant-operator-catalog-7fqc8"
Jul 09 15:10:44.563175 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.563139    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54nvd\" (UniqueName: \"kubernetes.io/projected/5fbdf502-f38a-41d7-9c02-a3d408c0e420-kube-api-access-54nvd\") pod \"kuadrant-operator-catalog-7fqc8\" (UID: \"5fbdf502-f38a-41d7-9c02-a3d408c0e420\") " pod="kuadrant-system/kuadrant-operator-catalog-7fqc8"
Jul 09 15:10:44.571915 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.571892    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54nvd\" (UniqueName: \"kubernetes.io/projected/5fbdf502-f38a-41d7-9c02-a3d408c0e420-kube-api-access-54nvd\") pod \"kuadrant-operator-catalog-7fqc8\" (UID: \"5fbdf502-f38a-41d7-9c02-a3d408c0e420\") " pod="kuadrant-system/kuadrant-operator-catalog-7fqc8"
Jul 09 15:10:44.654031 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.653930    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7fqc8"
Jul 09 15:10:44.698222 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.698191    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7fqc8"]
Jul 09 15:10:44.778461 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.778435    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7fqc8"]
Jul 09 15:10:44.780539 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:10:44.780515    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fbdf502_f38a_41d7_9c02_a3d408c0e420.slice/crio-84f4da68b05f57364fc56bfd89bc909d4d5db82ebd97448d3470fc0bf1447994 WatchSource:0}: Error finding container 84f4da68b05f57364fc56bfd89bc909d4d5db82ebd97448d3470fc0bf1447994: Status 404 returned error can't find the container with id 84f4da68b05f57364fc56bfd89bc909d4d5db82ebd97448d3470fc0bf1447994
Jul 09 15:10:44.910992 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.910917    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8kqc4"]
Jul 09 15:10:44.913444 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.913428    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8kqc4"
Jul 09 15:10:44.923576 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:44.923547    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8kqc4"]
Jul 09 15:10:45.005889 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:45.005857    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7fqc8" event={"ID":"5fbdf502-f38a-41d7-9c02-a3d408c0e420","Type":"ContainerStarted","Data":"84f4da68b05f57364fc56bfd89bc909d4d5db82ebd97448d3470fc0bf1447994"}
Jul 09 15:10:45.067427 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:45.067374    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxwnt\" (UniqueName: \"kubernetes.io/projected/82250586-da6f-4184-8741-2f6ae40f638c-kube-api-access-cxwnt\") pod \"kuadrant-operator-catalog-8kqc4\" (UID: \"82250586-da6f-4184-8741-2f6ae40f638c\") " pod="kuadrant-system/kuadrant-operator-catalog-8kqc4"
Jul 09 15:10:45.168435 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:45.168403    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxwnt\" (UniqueName: \"kubernetes.io/projected/82250586-da6f-4184-8741-2f6ae40f638c-kube-api-access-cxwnt\") pod \"kuadrant-operator-catalog-8kqc4\" (UID: \"82250586-da6f-4184-8741-2f6ae40f638c\") " pod="kuadrant-system/kuadrant-operator-catalog-8kqc4"
Jul 09 15:10:45.179985 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:45.179959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxwnt\" (UniqueName: \"kubernetes.io/projected/82250586-da6f-4184-8741-2f6ae40f638c-kube-api-access-cxwnt\") pod \"kuadrant-operator-catalog-8kqc4\" (UID: \"82250586-da6f-4184-8741-2f6ae40f638c\") " pod="kuadrant-system/kuadrant-operator-catalog-8kqc4"
Jul 09 15:10:45.221954 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:45.221924    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8kqc4"
Jul 09 15:10:45.366282 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:45.366248    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8kqc4"]
Jul 09 15:10:45.369039 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:10:45.369011    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82250586_da6f_4184_8741_2f6ae40f638c.slice/crio-adfffcaebbf32046da2a6bb7b8041857e6c4bd61bad71e10039775066e93e273 WatchSource:0}: Error finding container adfffcaebbf32046da2a6bb7b8041857e6c4bd61bad71e10039775066e93e273: Status 404 returned error can't find the container with id adfffcaebbf32046da2a6bb7b8041857e6c4bd61bad71e10039775066e93e273
Jul 09 15:10:46.011053 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:46.011010    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8kqc4" event={"ID":"82250586-da6f-4184-8741-2f6ae40f638c","Type":"ContainerStarted","Data":"adfffcaebbf32046da2a6bb7b8041857e6c4bd61bad71e10039775066e93e273"}
Jul 09 15:10:48.018852 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:48.018807    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7fqc8" event={"ID":"5fbdf502-f38a-41d7-9c02-a3d408c0e420","Type":"ContainerStarted","Data":"2c58a21aebfef644392dde46f027e6a10b4e9f5f4b05105f83e0d27de3bd63ae"}
Jul 09 15:10:48.019312 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:48.018870    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-7fqc8" podUID="5fbdf502-f38a-41d7-9c02-a3d408c0e420" containerName="registry-server" containerID="cri-o://2c58a21aebfef644392dde46f027e6a10b4e9f5f4b05105f83e0d27de3bd63ae" gracePeriod=2
Jul 09 15:10:48.020349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:48.020326    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8kqc4" event={"ID":"82250586-da6f-4184-8741-2f6ae40f638c","Type":"ContainerStarted","Data":"83b5d1e12a1f7ed0109ee9e1121add679d9551d4f327ddf44fee25db8ad58791"}
Jul 09 15:10:48.039047 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:48.039000    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7fqc8" podStartSLOduration=1.835288935 podStartE2EDuration="4.038982433s" totalImagesPullingTime="2.203693498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:10:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:10:48.037601413 +0000 UTC m=+498.371031376" watchObservedRunningTime="2026-07-09 15:10:48.038982433 +0000 UTC m=+498.372412405"
Jul 09 15:10:48.062716 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:48.061390    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-8kqc4" podStartSLOduration=2.44083182 podStartE2EDuration="4.061369636s" totalImagesPullingTime="1.620537816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:10:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:10:48.058601559 +0000 UTC m=+498.392031536" watchObservedRunningTime="2026-07-09 15:10:48.061369636 +0000 UTC m=+498.394799602"
Jul 09 15:10:48.252209 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:48.252185    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7fqc8"
Jul 09 15:10:48.397562 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:48.397464    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54nvd\" (UniqueName: \"kubernetes.io/projected/5fbdf502-f38a-41d7-9c02-a3d408c0e420-kube-api-access-54nvd\") pod \"5fbdf502-f38a-41d7-9c02-a3d408c0e420\" (UID: \"5fbdf502-f38a-41d7-9c02-a3d408c0e420\") "
Jul 09 15:10:48.399735 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:48.399699    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fbdf502-f38a-41d7-9c02-a3d408c0e420-kube-api-access-54nvd" (OuterVolumeSpecName: "kube-api-access-54nvd") pod "5fbdf502-f38a-41d7-9c02-a3d408c0e420" (UID: "5fbdf502-f38a-41d7-9c02-a3d408c0e420"). InnerVolumeSpecName "kube-api-access-54nvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:10:48.498282 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:48.498237    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54nvd\" (UniqueName: \"kubernetes.io/projected/5fbdf502-f38a-41d7-9c02-a3d408c0e420-kube-api-access-54nvd\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:10:49.025218 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:49.025184    2579 generic.go:356] "Generic (PLEG): container finished" podID="5fbdf502-f38a-41d7-9c02-a3d408c0e420" containerID="2c58a21aebfef644392dde46f027e6a10b4e9f5f4b05105f83e0d27de3bd63ae" exitCode=0
Jul 09 15:10:49.025607 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:49.025227    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7fqc8"
Jul 09 15:10:49.025607 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:49.025267    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7fqc8" event={"ID":"5fbdf502-f38a-41d7-9c02-a3d408c0e420","Type":"ContainerDied","Data":"2c58a21aebfef644392dde46f027e6a10b4e9f5f4b05105f83e0d27de3bd63ae"}
Jul 09 15:10:49.025607 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:49.025314    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7fqc8" event={"ID":"5fbdf502-f38a-41d7-9c02-a3d408c0e420","Type":"ContainerDied","Data":"84f4da68b05f57364fc56bfd89bc909d4d5db82ebd97448d3470fc0bf1447994"}
Jul 09 15:10:49.025607 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:49.025333    2579 scope.go:117] "RemoveContainer" containerID="2c58a21aebfef644392dde46f027e6a10b4e9f5f4b05105f83e0d27de3bd63ae"
Jul 09 15:10:49.033265 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:49.033251    2579 scope.go:117] "RemoveContainer" containerID="2c58a21aebfef644392dde46f027e6a10b4e9f5f4b05105f83e0d27de3bd63ae"
Jul 09 15:10:49.033504 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:10:49.033487    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c58a21aebfef644392dde46f027e6a10b4e9f5f4b05105f83e0d27de3bd63ae\": container with ID starting with 2c58a21aebfef644392dde46f027e6a10b4e9f5f4b05105f83e0d27de3bd63ae not found: ID does not exist" containerID="2c58a21aebfef644392dde46f027e6a10b4e9f5f4b05105f83e0d27de3bd63ae"
Jul 09 15:10:49.033570 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:49.033511    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c58a21aebfef644392dde46f027e6a10b4e9f5f4b05105f83e0d27de3bd63ae"} err="failed to get container status \"2c58a21aebfef644392dde46f027e6a10b4e9f5f4b05105f83e0d27de3bd63ae\": rpc error: code = NotFound desc = could not find container \"2c58a21aebfef644392dde46f027e6a10b4e9f5f4b05105f83e0d27de3bd63ae\": container with ID starting with 2c58a21aebfef644392dde46f027e6a10b4e9f5f4b05105f83e0d27de3bd63ae not found: ID does not exist"
Jul 09 15:10:49.046977 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:49.046952    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7fqc8"]
Jul 09 15:10:49.053028 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:49.053005    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7fqc8"]
Jul 09 15:10:50.229669 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:50.229637    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fbdf502-f38a-41d7-9c02-a3d408c0e420" path="/var/lib/kubelet/pods/5fbdf502-f38a-41d7-9c02-a3d408c0e420/volumes"
Jul 09 15:10:55.222402 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:55.222368    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-8kqc4"
Jul 09 15:10:55.222868 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:55.222628    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-8kqc4"
Jul 09 15:10:55.244357 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:55.244335    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-8kqc4"
Jul 09 15:10:56.071331 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:10:56.071307    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-8kqc4"
Jul 09 15:11:00.366664 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.366632    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"]
Jul 09 15:11:00.367071 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.366898    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fbdf502-f38a-41d7-9c02-a3d408c0e420" containerName="registry-server"
Jul 09 15:11:00.367071 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.366909    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fbdf502-f38a-41d7-9c02-a3d408c0e420" containerName="registry-server"
Jul 09 15:11:00.367071 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.366955    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fbdf502-f38a-41d7-9c02-a3d408c0e420" containerName="registry-server"
Jul 09 15:11:00.370031 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.370015    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"
Jul 09 15:11:00.373128 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.373107    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-czgkk\""
Jul 09 15:11:00.383133 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.383109    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"]
Jul 09 15:11:00.481952 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.481912    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9f891183-ad02-4bcf-85a7-18523561eb65-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7\" (UID: \"9f891183-ad02-4bcf-85a7-18523561eb65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"
Jul 09 15:11:00.482126 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.481970    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9f891183-ad02-4bcf-85a7-18523561eb65-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7\" (UID: \"9f891183-ad02-4bcf-85a7-18523561eb65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"
Jul 09 15:11:00.482126 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.481997    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldckp\" (UniqueName: \"kubernetes.io/projected/9f891183-ad02-4bcf-85a7-18523561eb65-kube-api-access-ldckp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7\" (UID: \"9f891183-ad02-4bcf-85a7-18523561eb65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"
Jul 09 15:11:00.582553 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.582508    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9f891183-ad02-4bcf-85a7-18523561eb65-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7\" (UID: \"9f891183-ad02-4bcf-85a7-18523561eb65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"
Jul 09 15:11:00.582764 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.582570    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9f891183-ad02-4bcf-85a7-18523561eb65-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7\" (UID: \"9f891183-ad02-4bcf-85a7-18523561eb65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"
Jul 09 15:11:00.582764 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.582598    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldckp\" (UniqueName: \"kubernetes.io/projected/9f891183-ad02-4bcf-85a7-18523561eb65-kube-api-access-ldckp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7\" (UID: \"9f891183-ad02-4bcf-85a7-18523561eb65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"
Jul 09 15:11:00.582916 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.582897    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9f891183-ad02-4bcf-85a7-18523561eb65-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7\" (UID: \"9f891183-ad02-4bcf-85a7-18523561eb65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"
Jul 09 15:11:00.582952 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.582926    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9f891183-ad02-4bcf-85a7-18523561eb65-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7\" (UID: \"9f891183-ad02-4bcf-85a7-18523561eb65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"
Jul 09 15:11:00.592120 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.592094    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldckp\" (UniqueName: \"kubernetes.io/projected/9f891183-ad02-4bcf-85a7-18523561eb65-kube-api-access-ldckp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7\" (UID: \"9f891183-ad02-4bcf-85a7-18523561eb65\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"
Jul 09 15:11:00.679076 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.679050    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"
Jul 09 15:11:00.789310 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.789281    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"]
Jul 09 15:11:00.793698 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.793668    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"
Jul 09 15:11:00.805577 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.805551    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"]
Jul 09 15:11:00.810420 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.810399    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"]
Jul 09 15:11:00.812588 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:11:00.812566    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f891183_ad02_4bcf_85a7_18523561eb65.slice/crio-3c5cc77a95c391e91cbcd6df4cc1b49a9dbd2ec1d7bbd0845b1dbc89df1b6c2e WatchSource:0}: Error finding container 3c5cc77a95c391e91cbcd6df4cc1b49a9dbd2ec1d7bbd0845b1dbc89df1b6c2e: Status 404 returned error can't find the container with id 3c5cc77a95c391e91cbcd6df4cc1b49a9dbd2ec1d7bbd0845b1dbc89df1b6c2e
Jul 09 15:11:00.885075 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.885038    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2af2c981-2b16-4b6b-9340-e7185e696eb9-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b\" (UID: \"2af2c981-2b16-4b6b-9340-e7185e696eb9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"
Jul 09 15:11:00.885227 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.885101    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2af2c981-2b16-4b6b-9340-e7185e696eb9-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b\" (UID: \"2af2c981-2b16-4b6b-9340-e7185e696eb9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"
Jul 09 15:11:00.885227 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.885149    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8m68\" (UniqueName: \"kubernetes.io/projected/2af2c981-2b16-4b6b-9340-e7185e696eb9-kube-api-access-x8m68\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b\" (UID: \"2af2c981-2b16-4b6b-9340-e7185e696eb9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"
Jul 09 15:11:00.985888 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.985815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2af2c981-2b16-4b6b-9340-e7185e696eb9-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b\" (UID: \"2af2c981-2b16-4b6b-9340-e7185e696eb9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"
Jul 09 15:11:00.986024 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.985890    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2af2c981-2b16-4b6b-9340-e7185e696eb9-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b\" (UID: \"2af2c981-2b16-4b6b-9340-e7185e696eb9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"
Jul 09 15:11:00.986024 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.985937    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8m68\" (UniqueName: \"kubernetes.io/projected/2af2c981-2b16-4b6b-9340-e7185e696eb9-kube-api-access-x8m68\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b\" (UID: \"2af2c981-2b16-4b6b-9340-e7185e696eb9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"
Jul 09 15:11:00.986244 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.986222    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2af2c981-2b16-4b6b-9340-e7185e696eb9-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b\" (UID: \"2af2c981-2b16-4b6b-9340-e7185e696eb9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"
Jul 09 15:11:00.986278 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.986227    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2af2c981-2b16-4b6b-9340-e7185e696eb9-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b\" (UID: \"2af2c981-2b16-4b6b-9340-e7185e696eb9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"
Jul 09 15:11:00.996938 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:00.996912    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8m68\" (UniqueName: \"kubernetes.io/projected/2af2c981-2b16-4b6b-9340-e7185e696eb9-kube-api-access-x8m68\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b\" (UID: \"2af2c981-2b16-4b6b-9340-e7185e696eb9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"
Jul 09 15:11:01.069357 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.069323    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7" event={"ID":"9f891183-ad02-4bcf-85a7-18523561eb65","Type":"ContainerStarted","Data":"3c5cc77a95c391e91cbcd6df4cc1b49a9dbd2ec1d7bbd0845b1dbc89df1b6c2e"}
Jul 09 15:11:01.102476 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.102444    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"
Jul 09 15:11:01.240843 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.240765    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"]
Jul 09 15:11:01.243581 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:11:01.243557    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af2c981_2b16_4b6b_9340_e7185e696eb9.slice/crio-4f5d0c8771c65675f6c3db116358fac265ff2ccafc94d250540d91570fbb31bb WatchSource:0}: Error finding container 4f5d0c8771c65675f6c3db116358fac265ff2ccafc94d250540d91570fbb31bb: Status 404 returned error can't find the container with id 4f5d0c8771c65675f6c3db116358fac265ff2ccafc94d250540d91570fbb31bb
Jul 09 15:11:01.394872 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.394834    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"]
Jul 09 15:11:01.399404 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.399381    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"
Jul 09 15:11:01.422585 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.422554    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"]
Jul 09 15:11:01.490564 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.490517    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d503cc81-1d1c-451f-ba63-e369839ba04e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg\" (UID: \"d503cc81-1d1c-451f-ba63-e369839ba04e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"
Jul 09 15:11:01.490564 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.490564    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d503cc81-1d1c-451f-ba63-e369839ba04e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg\" (UID: \"d503cc81-1d1c-451f-ba63-e369839ba04e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"
Jul 09 15:11:01.490806 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.490699    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhmn9\" (UniqueName: \"kubernetes.io/projected/d503cc81-1d1c-451f-ba63-e369839ba04e-kube-api-access-rhmn9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg\" (UID: \"d503cc81-1d1c-451f-ba63-e369839ba04e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"
Jul 09 15:11:01.592049 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.591964    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d503cc81-1d1c-451f-ba63-e369839ba04e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg\" (UID: \"d503cc81-1d1c-451f-ba63-e369839ba04e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"
Jul 09 15:11:01.592049 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.592005    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d503cc81-1d1c-451f-ba63-e369839ba04e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg\" (UID: \"d503cc81-1d1c-451f-ba63-e369839ba04e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"
Jul 09 15:11:01.592258 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.592067    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhmn9\" (UniqueName: \"kubernetes.io/projected/d503cc81-1d1c-451f-ba63-e369839ba04e-kube-api-access-rhmn9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg\" (UID: \"d503cc81-1d1c-451f-ba63-e369839ba04e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"
Jul 09 15:11:01.592457 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.592422    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d503cc81-1d1c-451f-ba63-e369839ba04e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg\" (UID: \"d503cc81-1d1c-451f-ba63-e369839ba04e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"
Jul 09 15:11:01.592585 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.592509    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d503cc81-1d1c-451f-ba63-e369839ba04e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg\" (UID: \"d503cc81-1d1c-451f-ba63-e369839ba04e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"
Jul 09 15:11:01.601924 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.601867    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhmn9\" (UniqueName: \"kubernetes.io/projected/d503cc81-1d1c-451f-ba63-e369839ba04e-kube-api-access-rhmn9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg\" (UID: \"d503cc81-1d1c-451f-ba63-e369839ba04e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"
Jul 09 15:11:01.707990 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.707952    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"
Jul 09 15:11:01.857214 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.857038    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"]
Jul 09 15:11:01.860458 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:11:01.860424    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd503cc81_1d1c_451f_ba63_e369839ba04e.slice/crio-37157d5a8dc584f367c0d7e7a1c31fc9d618db01fa850e41f43b4f496b4f84d4 WatchSource:0}: Error finding container 37157d5a8dc584f367c0d7e7a1c31fc9d618db01fa850e41f43b4f496b4f84d4: Status 404 returned error can't find the container with id 37157d5a8dc584f367c0d7e7a1c31fc9d618db01fa850e41f43b4f496b4f84d4
Jul 09 15:11:01.976519 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.976485    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"]
Jul 09 15:11:01.981071 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.981048    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"
Jul 09 15:11:01.992365 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:01.992340    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"]
Jul 09 15:11:02.075471 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:02.075432    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg" event={"ID":"d503cc81-1d1c-451f-ba63-e369839ba04e","Type":"ContainerStarted","Data":"37157d5a8dc584f367c0d7e7a1c31fc9d618db01fa850e41f43b4f496b4f84d4"}
Jul 09 15:11:02.076993 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:02.076965    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b" event={"ID":"2af2c981-2b16-4b6b-9340-e7185e696eb9","Type":"ContainerStarted","Data":"4f5d0c8771c65675f6c3db116358fac265ff2ccafc94d250540d91570fbb31bb"}
Jul 09 15:11:02.095561 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:02.095522    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb\" (UID: \"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"
Jul 09 15:11:02.095561 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:02.095563    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh95x\" (UniqueName: \"kubernetes.io/projected/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-kube-api-access-zh95x\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb\" (UID: \"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"
Jul 09 15:11:02.095801 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:02.095664    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb\" (UID: \"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"
Jul 09 15:11:02.196889 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:02.196854    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb\" (UID: \"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"
Jul 09 15:11:02.197081 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:02.196900    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh95x\" (UniqueName: \"kubernetes.io/projected/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-kube-api-access-zh95x\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb\" (UID: \"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"
Jul 09 15:11:02.197081 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:02.196980    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb\" (UID: \"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"
Jul 09 15:11:02.197321 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:02.197291    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb\" (UID: \"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"
Jul 09 15:11:02.197321 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:02.197315    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb\" (UID: \"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"
Jul 09 15:11:02.206998 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:02.206962    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh95x\" (UniqueName: \"kubernetes.io/projected/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-kube-api-access-zh95x\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb\" (UID: \"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"
Jul 09 15:11:02.290555 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:02.290293    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"
Jul 09 15:11:02.470331 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:02.470201    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"]
Jul 09 15:11:02.474326 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:11:02.474285    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5fe34af_cfa8_4342_a611_4e4a1bb9c21e.slice/crio-4c0bd50fa944e36da8b7021c6c9b9712a51b5cd3233a3c533ce8b226f3c60ba9 WatchSource:0}: Error finding container 4c0bd50fa944e36da8b7021c6c9b9712a51b5cd3233a3c533ce8b226f3c60ba9: Status 404 returned error can't find the container with id 4c0bd50fa944e36da8b7021c6c9b9712a51b5cd3233a3c533ce8b226f3c60ba9
Jul 09 15:11:03.084677 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:03.084637    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb" event={"ID":"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e","Type":"ContainerStarted","Data":"4c0bd50fa944e36da8b7021c6c9b9712a51b5cd3233a3c533ce8b226f3c60ba9"}
Jul 09 15:11:10.121337 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:10.121304    2579 generic.go:356] "Generic (PLEG): container finished" podID="d5fe34af-cfa8-4342-a611-4e4a1bb9c21e" containerID="f6e1a36dab2db8c99c1cb32027dfbed825bc68a6b2c79baee65180e31feb9e8e" exitCode=0
Jul 09 15:11:10.121764 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:10.121390    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb" event={"ID":"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e","Type":"ContainerDied","Data":"f6e1a36dab2db8c99c1cb32027dfbed825bc68a6b2c79baee65180e31feb9e8e"}
Jul 09 15:11:10.122883 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:10.122860    2579 generic.go:356] "Generic (PLEG): container finished" podID="9f891183-ad02-4bcf-85a7-18523561eb65" containerID="9ff958bc2d4f69a061c3ce69f15c782862720c350fa67e5df8510efd32cd7e83" exitCode=0
Jul 09 15:11:10.122987 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:10.122926    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7" event={"ID":"9f891183-ad02-4bcf-85a7-18523561eb65","Type":"ContainerDied","Data":"9ff958bc2d4f69a061c3ce69f15c782862720c350fa67e5df8510efd32cd7e83"}
Jul 09 15:11:10.124229 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:10.124210    2579 generic.go:356] "Generic (PLEG): container finished" podID="2af2c981-2b16-4b6b-9340-e7185e696eb9" containerID="6eb139ffeec3e8af6e985514c1597b4fb668603adf89a2faa52aef10dc75dad8" exitCode=0
Jul 09 15:11:10.124314 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:10.124291    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b" event={"ID":"2af2c981-2b16-4b6b-9340-e7185e696eb9","Type":"ContainerDied","Data":"6eb139ffeec3e8af6e985514c1597b4fb668603adf89a2faa52aef10dc75dad8"}
Jul 09 15:11:10.125715 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:10.125696    2579 generic.go:356] "Generic (PLEG): container finished" podID="d503cc81-1d1c-451f-ba63-e369839ba04e" containerID="1a255174fa42dac3cc624988fb4a4f6d21a82c1b1c66f69b1136c63465d23cdc" exitCode=0
Jul 09 15:11:10.125800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:10.125723    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg" event={"ID":"d503cc81-1d1c-451f-ba63-e369839ba04e","Type":"ContainerDied","Data":"1a255174fa42dac3cc624988fb4a4f6d21a82c1b1c66f69b1136c63465d23cdc"}
Jul 09 15:11:12.134974 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:12.134894    2579 generic.go:356] "Generic (PLEG): container finished" podID="d5fe34af-cfa8-4342-a611-4e4a1bb9c21e" containerID="44b45be84d89faabf2d198fa703e48bfdf0da011b3d554d410f83a8a3207d520" exitCode=0
Jul 09 15:11:12.135382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:12.134981    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb" event={"ID":"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e","Type":"ContainerDied","Data":"44b45be84d89faabf2d198fa703e48bfdf0da011b3d554d410f83a8a3207d520"}
Jul 09 15:11:12.137342 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:12.136972    2579 generic.go:356] "Generic (PLEG): container finished" podID="9f891183-ad02-4bcf-85a7-18523561eb65" containerID="3187ae19bdb1f9d5337d5f91d288692dab6d7cbbe66fbe3b0936501333c90d73" exitCode=0
Jul 09 15:11:12.137342 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:12.137091    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7" event={"ID":"9f891183-ad02-4bcf-85a7-18523561eb65","Type":"ContainerDied","Data":"3187ae19bdb1f9d5337d5f91d288692dab6d7cbbe66fbe3b0936501333c90d73"}
Jul 09 15:11:12.139312 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:12.139290    2579 generic.go:356] "Generic (PLEG): container finished" podID="d503cc81-1d1c-451f-ba63-e369839ba04e" containerID="d8596d454268dd311e6f6230081ae6c5f3357ea8450b57629a030009517e29d5" exitCode=0
Jul 09 15:11:12.139409 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:12.139337    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg" event={"ID":"d503cc81-1d1c-451f-ba63-e369839ba04e","Type":"ContainerDied","Data":"d8596d454268dd311e6f6230081ae6c5f3357ea8450b57629a030009517e29d5"}
Jul 09 15:11:13.148960 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:13.148341    2579 generic.go:356] "Generic (PLEG): container finished" podID="2af2c981-2b16-4b6b-9340-e7185e696eb9" containerID="c2a744a0a642671dfd488da08a85a8b19826b17f0755ba411e6c750df0c25643" exitCode=0
Jul 09 15:11:13.148960 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:13.148424    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b" event={"ID":"2af2c981-2b16-4b6b-9340-e7185e696eb9","Type":"ContainerDied","Data":"c2a744a0a642671dfd488da08a85a8b19826b17f0755ba411e6c750df0c25643"}
Jul 09 15:11:22.197494 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:22.197455    2579 generic.go:356] "Generic (PLEG): container finished" podID="9f891183-ad02-4bcf-85a7-18523561eb65" containerID="53605469155ab9f72e8a783a76cb9af7fcc5d9a401bf8432a2d7092f1bf5b954" exitCode=0
Jul 09 15:11:22.197932 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:22.197535    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7" event={"ID":"9f891183-ad02-4bcf-85a7-18523561eb65","Type":"ContainerDied","Data":"53605469155ab9f72e8a783a76cb9af7fcc5d9a401bf8432a2d7092f1bf5b954"}
Jul 09 15:11:22.199491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:22.199463    2579 generic.go:356] "Generic (PLEG): container finished" podID="2af2c981-2b16-4b6b-9340-e7185e696eb9" containerID="985442817c8127d09b5c8aeead936448ed2cd067279b05548ccd852adc834299" exitCode=0
Jul 09 15:11:22.199642 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:22.199540    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b" event={"ID":"2af2c981-2b16-4b6b-9340-e7185e696eb9","Type":"ContainerDied","Data":"985442817c8127d09b5c8aeead936448ed2cd067279b05548ccd852adc834299"}
Jul 09 15:11:22.201448 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:22.201429    2579 generic.go:356] "Generic (PLEG): container finished" podID="d503cc81-1d1c-451f-ba63-e369839ba04e" containerID="f62efee273c470bc0f6c22abdd673bf16057ee8eb379cfeea433a175d6111f98" exitCode=0
Jul 09 15:11:22.201555 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:22.201508    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg" event={"ID":"d503cc81-1d1c-451f-ba63-e369839ba04e","Type":"ContainerDied","Data":"f62efee273c470bc0f6c22abdd673bf16057ee8eb379cfeea433a175d6111f98"}
Jul 09 15:11:22.203464 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:22.203444    2579 generic.go:356] "Generic (PLEG): container finished" podID="d5fe34af-cfa8-4342-a611-4e4a1bb9c21e" containerID="d7b7428a0390867e84ccea904f973953f54d16b5969b905cafbb4ad924e20a1a" exitCode=0
Jul 09 15:11:22.203560 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:22.203495    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb" event={"ID":"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e","Type":"ContainerDied","Data":"d7b7428a0390867e84ccea904f973953f54d16b5969b905cafbb4ad924e20a1a"}
Jul 09 15:11:23.362784 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.362757    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"
Jul 09 15:11:23.393757 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.393728    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"
Jul 09 15:11:23.423138 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.423110    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"
Jul 09 15:11:23.431949 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.431925    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"
Jul 09 15:11:23.480214 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.480112    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhmn9\" (UniqueName: \"kubernetes.io/projected/d503cc81-1d1c-451f-ba63-e369839ba04e-kube-api-access-rhmn9\") pod \"d503cc81-1d1c-451f-ba63-e369839ba04e\" (UID: \"d503cc81-1d1c-451f-ba63-e369839ba04e\") "
Jul 09 15:11:23.480214 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.480168    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2af2c981-2b16-4b6b-9340-e7185e696eb9-util\") pod \"2af2c981-2b16-4b6b-9340-e7185e696eb9\" (UID: \"2af2c981-2b16-4b6b-9340-e7185e696eb9\") "
Jul 09 15:11:23.480214 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.480207    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2af2c981-2b16-4b6b-9340-e7185e696eb9-bundle\") pod \"2af2c981-2b16-4b6b-9340-e7185e696eb9\" (UID: \"2af2c981-2b16-4b6b-9340-e7185e696eb9\") "
Jul 09 15:11:23.480472 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.480239    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d503cc81-1d1c-451f-ba63-e369839ba04e-util\") pod \"d503cc81-1d1c-451f-ba63-e369839ba04e\" (UID: \"d503cc81-1d1c-451f-ba63-e369839ba04e\") "
Jul 09 15:11:23.480472 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.480264    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-util\") pod \"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e\" (UID: \"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e\") "
Jul 09 15:11:23.480472 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.480291    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d503cc81-1d1c-451f-ba63-e369839ba04e-bundle\") pod \"d503cc81-1d1c-451f-ba63-e369839ba04e\" (UID: \"d503cc81-1d1c-451f-ba63-e369839ba04e\") "
Jul 09 15:11:23.480472 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.480319    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldckp\" (UniqueName: \"kubernetes.io/projected/9f891183-ad02-4bcf-85a7-18523561eb65-kube-api-access-ldckp\") pod \"9f891183-ad02-4bcf-85a7-18523561eb65\" (UID: \"9f891183-ad02-4bcf-85a7-18523561eb65\") "
Jul 09 15:11:23.480472 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.480346    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zh95x\" (UniqueName: \"kubernetes.io/projected/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-kube-api-access-zh95x\") pod \"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e\" (UID: \"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e\") "
Jul 09 15:11:23.480472 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.480380    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-bundle\") pod \"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e\" (UID: \"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e\") "
Jul 09 15:11:23.480472 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.480408    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8m68\" (UniqueName: \"kubernetes.io/projected/2af2c981-2b16-4b6b-9340-e7185e696eb9-kube-api-access-x8m68\") pod \"2af2c981-2b16-4b6b-9340-e7185e696eb9\" (UID: \"2af2c981-2b16-4b6b-9340-e7185e696eb9\") "
Jul 09 15:11:23.480472 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.480431    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9f891183-ad02-4bcf-85a7-18523561eb65-bundle\") pod \"9f891183-ad02-4bcf-85a7-18523561eb65\" (UID: \"9f891183-ad02-4bcf-85a7-18523561eb65\") "
Jul 09 15:11:23.480871 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.480478    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9f891183-ad02-4bcf-85a7-18523561eb65-util\") pod \"9f891183-ad02-4bcf-85a7-18523561eb65\" (UID: \"9f891183-ad02-4bcf-85a7-18523561eb65\") "
Jul 09 15:11:23.481142 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.481104    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d503cc81-1d1c-451f-ba63-e369839ba04e-bundle" (OuterVolumeSpecName: "bundle") pod "d503cc81-1d1c-451f-ba63-e369839ba04e" (UID: "d503cc81-1d1c-451f-ba63-e369839ba04e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:11:23.482023 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.481981    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-bundle" (OuterVolumeSpecName: "bundle") pod "d5fe34af-cfa8-4342-a611-4e4a1bb9c21e" (UID: "d5fe34af-cfa8-4342-a611-4e4a1bb9c21e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:11:23.483151 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.483069    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d503cc81-1d1c-451f-ba63-e369839ba04e-kube-api-access-rhmn9" (OuterVolumeSpecName: "kube-api-access-rhmn9") pod "d503cc81-1d1c-451f-ba63-e369839ba04e" (UID: "d503cc81-1d1c-451f-ba63-e369839ba04e"). InnerVolumeSpecName "kube-api-access-rhmn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:11:23.484425 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.484390    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f891183-ad02-4bcf-85a7-18523561eb65-kube-api-access-ldckp" (OuterVolumeSpecName: "kube-api-access-ldckp") pod "9f891183-ad02-4bcf-85a7-18523561eb65" (UID: "9f891183-ad02-4bcf-85a7-18523561eb65"). InnerVolumeSpecName "kube-api-access-ldckp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:11:23.484977 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.484955    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2af2c981-2b16-4b6b-9340-e7185e696eb9-kube-api-access-x8m68" (OuterVolumeSpecName: "kube-api-access-x8m68") pod "2af2c981-2b16-4b6b-9340-e7185e696eb9" (UID: "2af2c981-2b16-4b6b-9340-e7185e696eb9"). InnerVolumeSpecName "kube-api-access-x8m68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:11:23.486214 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.486186    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f891183-ad02-4bcf-85a7-18523561eb65-bundle" (OuterVolumeSpecName: "bundle") pod "9f891183-ad02-4bcf-85a7-18523561eb65" (UID: "9f891183-ad02-4bcf-85a7-18523561eb65"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:11:23.486892 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.486867    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d503cc81-1d1c-451f-ba63-e369839ba04e-util" (OuterVolumeSpecName: "util") pod "d503cc81-1d1c-451f-ba63-e369839ba04e" (UID: "d503cc81-1d1c-451f-ba63-e369839ba04e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:11:23.487449 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.487424    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af2c981-2b16-4b6b-9340-e7185e696eb9-util" (OuterVolumeSpecName: "util") pod "2af2c981-2b16-4b6b-9340-e7185e696eb9" (UID: "2af2c981-2b16-4b6b-9340-e7185e696eb9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:11:23.487978 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.487952    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-kube-api-access-zh95x" (OuterVolumeSpecName: "kube-api-access-zh95x") pod "d5fe34af-cfa8-4342-a611-4e4a1bb9c21e" (UID: "d5fe34af-cfa8-4342-a611-4e4a1bb9c21e"). InnerVolumeSpecName "kube-api-access-zh95x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:11:23.488289 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.488269    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f891183-ad02-4bcf-85a7-18523561eb65-util" (OuterVolumeSpecName: "util") pod "9f891183-ad02-4bcf-85a7-18523561eb65" (UID: "9f891183-ad02-4bcf-85a7-18523561eb65"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:11:23.488430 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.488405    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-util" (OuterVolumeSpecName: "util") pod "d5fe34af-cfa8-4342-a611-4e4a1bb9c21e" (UID: "d5fe34af-cfa8-4342-a611-4e4a1bb9c21e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:11:23.488802 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.488779    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af2c981-2b16-4b6b-9340-e7185e696eb9-bundle" (OuterVolumeSpecName: "bundle") pod "2af2c981-2b16-4b6b-9340-e7185e696eb9" (UID: "2af2c981-2b16-4b6b-9340-e7185e696eb9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:11:23.581652 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.581619    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9f891183-ad02-4bcf-85a7-18523561eb65-util\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:11:23.581652 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.581651    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhmn9\" (UniqueName: \"kubernetes.io/projected/d503cc81-1d1c-451f-ba63-e369839ba04e-kube-api-access-rhmn9\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:11:23.581652 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.581667    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2af2c981-2b16-4b6b-9340-e7185e696eb9-util\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:11:23.581924 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.581679    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2af2c981-2b16-4b6b-9340-e7185e696eb9-bundle\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:11:23.581924 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.581713    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d503cc81-1d1c-451f-ba63-e369839ba04e-util\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:11:23.581924 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.581723    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-util\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:11:23.581924 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.581732    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d503cc81-1d1c-451f-ba63-e369839ba04e-bundle\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:11:23.581924 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.581743    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldckp\" (UniqueName: \"kubernetes.io/projected/9f891183-ad02-4bcf-85a7-18523561eb65-kube-api-access-ldckp\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:11:23.581924 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.581753    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh95x\" (UniqueName: \"kubernetes.io/projected/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-kube-api-access-zh95x\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:11:23.581924 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.581766    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5fe34af-cfa8-4342-a611-4e4a1bb9c21e-bundle\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:11:23.581924 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.581777    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8m68\" (UniqueName: \"kubernetes.io/projected/2af2c981-2b16-4b6b-9340-e7185e696eb9-kube-api-access-x8m68\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:11:23.581924 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:23.581787    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9f891183-ad02-4bcf-85a7-18523561eb65-bundle\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:11:24.213307 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:24.213283    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7"
Jul 09 15:11:24.213491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:24.213275    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7" event={"ID":"9f891183-ad02-4bcf-85a7-18523561eb65","Type":"ContainerDied","Data":"3c5cc77a95c391e91cbcd6df4cc1b49a9dbd2ec1d7bbd0845b1dbc89df1b6c2e"}
Jul 09 15:11:24.213491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:24.213438    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c5cc77a95c391e91cbcd6df4cc1b49a9dbd2ec1d7bbd0845b1dbc89df1b6c2e"
Jul 09 15:11:24.215068 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:24.215048    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b"
Jul 09 15:11:24.215202 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:24.215046    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b" event={"ID":"2af2c981-2b16-4b6b-9340-e7185e696eb9","Type":"ContainerDied","Data":"4f5d0c8771c65675f6c3db116358fac265ff2ccafc94d250540d91570fbb31bb"}
Jul 09 15:11:24.215267 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:24.215212    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f5d0c8771c65675f6c3db116358fac265ff2ccafc94d250540d91570fbb31bb"
Jul 09 15:11:24.216826 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:24.216808    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg"
Jul 09 15:11:24.216938 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:24.216826    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg" event={"ID":"d503cc81-1d1c-451f-ba63-e369839ba04e","Type":"ContainerDied","Data":"37157d5a8dc584f367c0d7e7a1c31fc9d618db01fa850e41f43b4f496b4f84d4"}
Jul 09 15:11:24.216938 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:24.216865    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37157d5a8dc584f367c0d7e7a1c31fc9d618db01fa850e41f43b4f496b4f84d4"
Jul 09 15:11:24.218756 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:24.218733    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb" event={"ID":"d5fe34af-cfa8-4342-a611-4e4a1bb9c21e","Type":"ContainerDied","Data":"4c0bd50fa944e36da8b7021c6c9b9712a51b5cd3233a3c533ce8b226f3c60ba9"}
Jul 09 15:11:24.218878 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:24.218766    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4c0bd50fa944e36da8b7021c6c9b9712a51b5cd3233a3c533ce8b226f3c60ba9"
Jul 09 15:11:24.218878 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:24.218777    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb"
Jul 09 15:11:32.910212 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910175    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-x296k"]
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910456    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d503cc81-1d1c-451f-ba63-e369839ba04e" containerName="util"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910467    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d503cc81-1d1c-451f-ba63-e369839ba04e" containerName="util"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910476    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d503cc81-1d1c-451f-ba63-e369839ba04e" containerName="extract"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910482    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d503cc81-1d1c-451f-ba63-e369839ba04e" containerName="extract"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910492    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f891183-ad02-4bcf-85a7-18523561eb65" containerName="pull"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910498    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f891183-ad02-4bcf-85a7-18523561eb65" containerName="pull"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910505    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5fe34af-cfa8-4342-a611-4e4a1bb9c21e" containerName="pull"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910510    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5fe34af-cfa8-4342-a611-4e4a1bb9c21e" containerName="pull"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910515    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2af2c981-2b16-4b6b-9340-e7185e696eb9" containerName="extract"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910520    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2af2c981-2b16-4b6b-9340-e7185e696eb9" containerName="extract"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910527    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2af2c981-2b16-4b6b-9340-e7185e696eb9" containerName="util"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910532    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2af2c981-2b16-4b6b-9340-e7185e696eb9" containerName="util"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910537    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f891183-ad02-4bcf-85a7-18523561eb65" containerName="extract"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910542    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f891183-ad02-4bcf-85a7-18523561eb65" containerName="extract"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910550    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5fe34af-cfa8-4342-a611-4e4a1bb9c21e" containerName="util"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910555    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5fe34af-cfa8-4342-a611-4e4a1bb9c21e" containerName="util"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910562    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2af2c981-2b16-4b6b-9340-e7185e696eb9" containerName="pull"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910567    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2af2c981-2b16-4b6b-9340-e7185e696eb9" containerName="pull"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910574    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f891183-ad02-4bcf-85a7-18523561eb65" containerName="util"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910579    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f891183-ad02-4bcf-85a7-18523561eb65" containerName="util"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910584    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d503cc81-1d1c-451f-ba63-e369839ba04e" containerName="pull"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910589    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d503cc81-1d1c-451f-ba63-e369839ba04e" containerName="pull"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910596    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5fe34af-cfa8-4342-a611-4e4a1bb9c21e" containerName="extract"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910602    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5fe34af-cfa8-4342-a611-4e4a1bb9c21e" containerName="extract"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910645    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2af2c981-2b16-4b6b-9340-e7185e696eb9" containerName="extract"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910651    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f891183-ad02-4bcf-85a7-18523561eb65" containerName="extract"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910658    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d503cc81-1d1c-451f-ba63-e369839ba04e" containerName="extract"
Jul 09 15:11:32.910653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.910664    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5fe34af-cfa8-4342-a611-4e4a1bb9c21e" containerName="extract"
Jul 09 15:11:32.913678 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.913658    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x296k"
Jul 09 15:11:32.917411 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.917382    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 09 15:11:32.917565 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.917382    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-dhxn6\""
Jul 09 15:11:32.930924 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.930896    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-x296k"]
Jul 09 15:11:32.951607 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:32.951567    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66d2m\" (UniqueName: \"kubernetes.io/projected/ee481a72-01ae-45ac-bdc9-9950abe4421f-kube-api-access-66d2m\") pod \"dns-operator-controller-manager-864dd78c4c-x296k\" (UID: \"ee481a72-01ae-45ac-bdc9-9950abe4421f\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x296k"
Jul 09 15:11:33.052607 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:33.052571    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66d2m\" (UniqueName: \"kubernetes.io/projected/ee481a72-01ae-45ac-bdc9-9950abe4421f-kube-api-access-66d2m\") pod \"dns-operator-controller-manager-864dd78c4c-x296k\" (UID: \"ee481a72-01ae-45ac-bdc9-9950abe4421f\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x296k"
Jul 09 15:11:33.062389 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:33.062358    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66d2m\" (UniqueName: \"kubernetes.io/projected/ee481a72-01ae-45ac-bdc9-9950abe4421f-kube-api-access-66d2m\") pod \"dns-operator-controller-manager-864dd78c4c-x296k\" (UID: \"ee481a72-01ae-45ac-bdc9-9950abe4421f\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x296k"
Jul 09 15:11:33.224133 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:33.224098    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x296k"
Jul 09 15:11:33.354343 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:33.354312    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-x296k"]
Jul 09 15:11:33.357344 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:11:33.357318    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee481a72_01ae_45ac_bdc9_9950abe4421f.slice/crio-c45b38fef03cca6a9317f106b70e1af46df42e7138d071f9bff64eeaac8b2a66 WatchSource:0}: Error finding container c45b38fef03cca6a9317f106b70e1af46df42e7138d071f9bff64eeaac8b2a66: Status 404 returned error can't find the container with id c45b38fef03cca6a9317f106b70e1af46df42e7138d071f9bff64eeaac8b2a66
Jul 09 15:11:34.259302 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:34.259269    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x296k" event={"ID":"ee481a72-01ae-45ac-bdc9-9950abe4421f","Type":"ContainerStarted","Data":"c45b38fef03cca6a9317f106b70e1af46df42e7138d071f9bff64eeaac8b2a66"}
Jul 09 15:11:36.269556 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:36.269461    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x296k" event={"ID":"ee481a72-01ae-45ac-bdc9-9950abe4421f","Type":"ContainerStarted","Data":"c8d0c2dcfe23a03108346866c4d9fca4941cdbe3120e4e8bd93085bb265940eb"}
Jul 09 15:11:36.269556 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:36.269526    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x296k"
Jul 09 15:11:36.289858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:36.289712    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x296k" podStartSLOduration=1.658517572 podStartE2EDuration="4.289673149s" totalImagesPullingTime="2.631155577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:11:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:11:36.289443611 +0000 UTC m=+546.622873577" watchObservedRunningTime="2026-07-09 15:11:36.289673149 +0000 UTC m=+546.623103115"
Jul 09 15:11:36.991497 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:36.991459    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-gf42b"]
Jul 09 15:11:36.994540 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:36.994523    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-gf42b"
Jul 09 15:11:36.997665 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:36.997643    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-m9c9l\""
Jul 09 15:11:37.005674 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:37.005647    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-gf42b"]
Jul 09 15:11:37.090243 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:37.090207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmc2z\" (UniqueName: \"kubernetes.io/projected/f8c4742e-2296-4bf5-a111-a2b828175a6f-kube-api-access-lmc2z\") pod \"authorino-operator-75cc8d5cb7-gf42b\" (UID: \"f8c4742e-2296-4bf5-a111-a2b828175a6f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-gf42b"
Jul 09 15:11:37.191465 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:37.191413    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmc2z\" (UniqueName: \"kubernetes.io/projected/f8c4742e-2296-4bf5-a111-a2b828175a6f-kube-api-access-lmc2z\") pod \"authorino-operator-75cc8d5cb7-gf42b\" (UID: \"f8c4742e-2296-4bf5-a111-a2b828175a6f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-gf42b"
Jul 09 15:11:37.213890 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:37.213862    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmc2z\" (UniqueName: \"kubernetes.io/projected/f8c4742e-2296-4bf5-a111-a2b828175a6f-kube-api-access-lmc2z\") pod \"authorino-operator-75cc8d5cb7-gf42b\" (UID: \"f8c4742e-2296-4bf5-a111-a2b828175a6f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-gf42b"
Jul 09 15:11:37.304504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:37.304407    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-gf42b"
Jul 09 15:11:37.443210 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:37.443167    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-gf42b"]
Jul 09 15:11:37.445295 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:11:37.445269    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c4742e_2296_4bf5_a111_a2b828175a6f.slice/crio-3210428e4d6be8f22dfa78b9ab8a5ee118508fd5c4a6658296dd9284b65f1b5a WatchSource:0}: Error finding container 3210428e4d6be8f22dfa78b9ab8a5ee118508fd5c4a6658296dd9284b65f1b5a: Status 404 returned error can't find the container with id 3210428e4d6be8f22dfa78b9ab8a5ee118508fd5c4a6658296dd9284b65f1b5a
Jul 09 15:11:38.282174 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:38.282136    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-gf42b" event={"ID":"f8c4742e-2296-4bf5-a111-a2b828175a6f","Type":"ContainerStarted","Data":"3210428e4d6be8f22dfa78b9ab8a5ee118508fd5c4a6658296dd9284b65f1b5a"}
Jul 09 15:11:40.291433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:40.291396    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"]
Jul 09 15:11:40.294852 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:40.294826    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-gf42b" event={"ID":"f8c4742e-2296-4bf5-a111-a2b828175a6f","Type":"ContainerStarted","Data":"e1d9e1e98985aed635b8a46a1047b269cfad4b2258a0e77cc3ef303e557a8a6d"}
Jul 09 15:11:40.294991 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:40.294862    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-gf42b"
Jul 09 15:11:40.294991 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:40.294921    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"
Jul 09 15:11:40.297516 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:40.297498    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-724bx\""
Jul 09 15:11:40.311800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:40.311772    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"]
Jul 09 15:11:40.333660 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:40.333600    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-gf42b" podStartSLOduration=1.895950085 podStartE2EDuration="4.333585939s" totalImagesPullingTime="2.437635854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:11:40.331240666 +0000 UTC m=+550.664670627" watchObservedRunningTime="2026-07-09 15:11:40.333585939 +0000 UTC m=+550.667015904"
Jul 09 15:11:40.418795 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:40.418747    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfjfl\" (UniqueName: \"kubernetes.io/projected/b6656694-4555-46fc-b822-5a279c511a13-kube-api-access-mfjfl\") pod \"limitador-operator-controller-manager-859d7b78bd-2q9b8\" (UID: \"b6656694-4555-46fc-b822-5a279c511a13\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"
Jul 09 15:11:40.519510 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:40.519456    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfjfl\" (UniqueName: \"kubernetes.io/projected/b6656694-4555-46fc-b822-5a279c511a13-kube-api-access-mfjfl\") pod \"limitador-operator-controller-manager-859d7b78bd-2q9b8\" (UID: \"b6656694-4555-46fc-b822-5a279c511a13\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"
Jul 09 15:11:40.532708 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:40.532652    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfjfl\" (UniqueName: \"kubernetes.io/projected/b6656694-4555-46fc-b822-5a279c511a13-kube-api-access-mfjfl\") pod \"limitador-operator-controller-manager-859d7b78bd-2q9b8\" (UID: \"b6656694-4555-46fc-b822-5a279c511a13\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"
Jul 09 15:11:40.604232 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:40.604134    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"
Jul 09 15:11:40.760631 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:40.760606    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"]
Jul 09 15:11:40.762894 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:11:40.762868    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6656694_4555_46fc_b822_5a279c511a13.slice/crio-1caaf3dafe167e1691dc25afc91f7d4997f4ae499f24a601e4a3f5bcad0e4b28 WatchSource:0}: Error finding container 1caaf3dafe167e1691dc25afc91f7d4997f4ae499f24a601e4a3f5bcad0e4b28: Status 404 returned error can't find the container with id 1caaf3dafe167e1691dc25afc91f7d4997f4ae499f24a601e4a3f5bcad0e4b28
Jul 09 15:11:41.299516 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:41.299480    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8" event={"ID":"b6656694-4555-46fc-b822-5a279c511a13","Type":"ContainerStarted","Data":"1caaf3dafe167e1691dc25afc91f7d4997f4ae499f24a601e4a3f5bcad0e4b28"}
Jul 09 15:11:43.310264 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:43.310228    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8" event={"ID":"b6656694-4555-46fc-b822-5a279c511a13","Type":"ContainerStarted","Data":"ac0178898ed27a9bc132bede7aa086cfc1a2e478c797853c4d609799b5e7b99c"}
Jul 09 15:11:43.310651 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:43.310313    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"
Jul 09 15:11:43.341829 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:43.341762    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8" podStartSLOduration=1.4940581050000001 podStartE2EDuration="3.341742818s" totalImagesPullingTime="1.847684713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:11:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:11:43.340297551 +0000 UTC m=+553.673727516" watchObservedRunningTime="2026-07-09 15:11:43.341742818 +0000 UTC m=+553.675172783"
Jul 09 15:11:47.276060 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:47.276029    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-x296k"
Jul 09 15:11:51.302151 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:51.302120    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-gf42b"
Jul 09 15:11:54.316733 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:54.316668    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"
Jul 09 15:11:55.564434 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:55.564397    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t"]
Jul 09 15:11:55.566536 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:55.566517    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t"
Jul 09 15:11:55.571678 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:55.571653    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 09 15:11:55.571949 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:55.571932    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 09 15:11:55.572840 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:55.572462    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-czgkk\""
Jul 09 15:11:55.581499 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:55.581470    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t"]
Jul 09 15:11:55.638764 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:55.638723    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/646260f4-cd3e-4a8e-855b-71d52fe2b470-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gwz7t\" (UID: \"646260f4-cd3e-4a8e-855b-71d52fe2b470\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t"
Jul 09 15:11:55.638764 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:55.638767    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96cgj\" (UniqueName: \"kubernetes.io/projected/646260f4-cd3e-4a8e-855b-71d52fe2b470-kube-api-access-96cgj\") pod \"kuadrant-console-plugin-66b5774985-gwz7t\" (UID: \"646260f4-cd3e-4a8e-855b-71d52fe2b470\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t"
Jul 09 15:11:55.638977 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:55.638845    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/646260f4-cd3e-4a8e-855b-71d52fe2b470-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gwz7t\" (UID: \"646260f4-cd3e-4a8e-855b-71d52fe2b470\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t"
Jul 09 15:11:55.739747 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:55.739681    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96cgj\" (UniqueName: \"kubernetes.io/projected/646260f4-cd3e-4a8e-855b-71d52fe2b470-kube-api-access-96cgj\") pod \"kuadrant-console-plugin-66b5774985-gwz7t\" (UID: \"646260f4-cd3e-4a8e-855b-71d52fe2b470\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t"
Jul 09 15:11:55.739955 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:55.739766    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/646260f4-cd3e-4a8e-855b-71d52fe2b470-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gwz7t\" (UID: \"646260f4-cd3e-4a8e-855b-71d52fe2b470\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t"
Jul 09 15:11:55.739955 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:55.739816    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/646260f4-cd3e-4a8e-855b-71d52fe2b470-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gwz7t\" (UID: \"646260f4-cd3e-4a8e-855b-71d52fe2b470\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t"
Jul 09 15:11:55.739955 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:11:55.739916    2579 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 09 15:11:55.740105 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:11:55.739983    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/646260f4-cd3e-4a8e-855b-71d52fe2b470-plugin-serving-cert podName:646260f4-cd3e-4a8e-855b-71d52fe2b470 nodeName:}" failed. No retries permitted until 2026-07-09 15:11:56.239962705 +0000 UTC m=+566.573392656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/646260f4-cd3e-4a8e-855b-71d52fe2b470-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-gwz7t" (UID: "646260f4-cd3e-4a8e-855b-71d52fe2b470") : secret "plugin-serving-cert" not found
Jul 09 15:11:55.740504 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:55.740484    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/646260f4-cd3e-4a8e-855b-71d52fe2b470-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gwz7t\" (UID: \"646260f4-cd3e-4a8e-855b-71d52fe2b470\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t"
Jul 09 15:11:55.759027 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:55.759001    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96cgj\" (UniqueName: \"kubernetes.io/projected/646260f4-cd3e-4a8e-855b-71d52fe2b470-kube-api-access-96cgj\") pod \"kuadrant-console-plugin-66b5774985-gwz7t\" (UID: \"646260f4-cd3e-4a8e-855b-71d52fe2b470\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t"
Jul 09 15:11:56.245059 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:56.245027    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/646260f4-cd3e-4a8e-855b-71d52fe2b470-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gwz7t\" (UID: \"646260f4-cd3e-4a8e-855b-71d52fe2b470\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t"
Jul 09 15:11:56.247435 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:56.247405    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/646260f4-cd3e-4a8e-855b-71d52fe2b470-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gwz7t\" (UID: \"646260f4-cd3e-4a8e-855b-71d52fe2b470\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t"
Jul 09 15:11:56.474899 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:56.474861    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t"
Jul 09 15:11:56.615848 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:56.615814    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t"]
Jul 09 15:11:56.618280 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:11:56.618252    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod646260f4_cd3e_4a8e_855b_71d52fe2b470.slice/crio-73d5a71584f7bd8d90ab5ed161e0c436ceb14e6c7d32c2cb051739f1acb563a9 WatchSource:0}: Error finding container 73d5a71584f7bd8d90ab5ed161e0c436ceb14e6c7d32c2cb051739f1acb563a9: Status 404 returned error can't find the container with id 73d5a71584f7bd8d90ab5ed161e0c436ceb14e6c7d32c2cb051739f1acb563a9
Jul 09 15:11:57.372223 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:11:57.372189    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t" event={"ID":"646260f4-cd3e-4a8e-855b-71d52fe2b470","Type":"ContainerStarted","Data":"73d5a71584f7bd8d90ab5ed161e0c436ceb14e6c7d32c2cb051739f1acb563a9"}
Jul 09 15:12:06.645893 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:06.645851    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"]
Jul 09 15:12:06.646462 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:06.646099    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8" podUID="b6656694-4555-46fc-b822-5a279c511a13" containerName="manager" containerID="cri-o://ac0178898ed27a9bc132bede7aa086cfc1a2e478c797853c4d609799b5e7b99c" gracePeriod=2
Jul 09 15:12:06.652449 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:06.652353    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"]
Jul 09 15:12:06.712312 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:06.712267    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnx2c"]
Jul 09 15:12:06.712748 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:06.712730    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6656694-4555-46fc-b822-5a279c511a13" containerName="manager"
Jul 09 15:12:06.712857 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:06.712750    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6656694-4555-46fc-b822-5a279c511a13" containerName="manager"
Jul 09 15:12:06.712857 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:06.712828    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6656694-4555-46fc-b822-5a279c511a13" containerName="manager"
Jul 09 15:12:06.720989 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:06.720956    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnx2c"
Jul 09 15:12:06.724536 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:12:06.724493    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2q9b8\" is forbidden: User \"system:node:ip-10-0-129-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-152.ec2.internal' and this object" podUID="b6656694-4555-46fc-b822-5a279c511a13" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"
Jul 09 15:12:06.744703 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:06.744652    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnx2c"]
Jul 09 15:12:06.836382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:06.836337    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-275nv\" (UniqueName: \"kubernetes.io/projected/ae42b068-625d-4729-bf26-8468ea371a8f-kube-api-access-275nv\") pod \"limitador-operator-controller-manager-859d7b78bd-hnx2c\" (UID: \"ae42b068-625d-4729-bf26-8468ea371a8f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnx2c"
Jul 09 15:12:06.937582 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:06.937352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-275nv\" (UniqueName: \"kubernetes.io/projected/ae42b068-625d-4729-bf26-8468ea371a8f-kube-api-access-275nv\") pod \"limitador-operator-controller-manager-859d7b78bd-hnx2c\" (UID: \"ae42b068-625d-4729-bf26-8468ea371a8f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnx2c"
Jul 09 15:12:06.955743 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:06.955707    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-275nv\" (UniqueName: \"kubernetes.io/projected/ae42b068-625d-4729-bf26-8468ea371a8f-kube-api-access-275nv\") pod \"limitador-operator-controller-manager-859d7b78bd-hnx2c\" (UID: \"ae42b068-625d-4729-bf26-8468ea371a8f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnx2c"
Jul 09 15:12:07.033747 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:07.033704    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnx2c"
Jul 09 15:12:19.748005 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:19.747974    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"
Jul 09 15:12:19.751162 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:12:19.751121    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2q9b8\" is forbidden: User \"system:node:ip-10-0-129-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-152.ec2.internal' and this object" podUID="b6656694-4555-46fc-b822-5a279c511a13" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"
Jul 09 15:12:19.759192 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:19.759163    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnx2c"]
Jul 09 15:12:19.763502 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:12:19.763468    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae42b068_625d_4729_bf26_8468ea371a8f.slice/crio-6b2eba9a05af70d3a5cce2c72904e8997be93047a4e1596902fd7c1a846cdd1e WatchSource:0}: Error finding container 6b2eba9a05af70d3a5cce2c72904e8997be93047a4e1596902fd7c1a846cdd1e: Status 404 returned error can't find the container with id 6b2eba9a05af70d3a5cce2c72904e8997be93047a4e1596902fd7c1a846cdd1e
Jul 09 15:12:19.844503 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:19.844461    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfjfl\" (UniqueName: \"kubernetes.io/projected/b6656694-4555-46fc-b822-5a279c511a13-kube-api-access-mfjfl\") pod \"b6656694-4555-46fc-b822-5a279c511a13\" (UID: \"b6656694-4555-46fc-b822-5a279c511a13\") "
Jul 09 15:12:19.846720 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:19.846671    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6656694-4555-46fc-b822-5a279c511a13-kube-api-access-mfjfl" (OuterVolumeSpecName: "kube-api-access-mfjfl") pod "b6656694-4555-46fc-b822-5a279c511a13" (UID: "b6656694-4555-46fc-b822-5a279c511a13"). InnerVolumeSpecName "kube-api-access-mfjfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:12:19.945385 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:19.945341    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfjfl\" (UniqueName: \"kubernetes.io/projected/b6656694-4555-46fc-b822-5a279c511a13-kube-api-access-mfjfl\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:12:20.229963 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:12:20.229858    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2q9b8\" is forbidden: User \"system:node:ip-10-0-129-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-152.ec2.internal' and this object" podUID="b6656694-4555-46fc-b822-5a279c511a13" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"
Jul 09 15:12:20.231008 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:20.230986    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6656694-4555-46fc-b822-5a279c511a13" path="/var/lib/kubelet/pods/b6656694-4555-46fc-b822-5a279c511a13/volumes"
Jul 09 15:12:20.491509 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:20.491402    2579 generic.go:356] "Generic (PLEG): container finished" podID="b6656694-4555-46fc-b822-5a279c511a13" containerID="ac0178898ed27a9bc132bede7aa086cfc1a2e478c797853c4d609799b5e7b99c" exitCode=0
Jul 09 15:12:20.491509 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:20.491448    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"
Jul 09 15:12:20.491797 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:20.491509    2579 scope.go:117] "RemoveContainer" containerID="ac0178898ed27a9bc132bede7aa086cfc1a2e478c797853c4d609799b5e7b99c"
Jul 09 15:12:20.493338 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:20.493269    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t" event={"ID":"646260f4-cd3e-4a8e-855b-71d52fe2b470","Type":"ContainerStarted","Data":"b1df09399772896377f814e2ea17d50bbe2b5c0c58116e0712ef01031c8b9452"}
Jul 09 15:12:20.494131 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:12:20.494085    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2q9b8\" is forbidden: User \"system:node:ip-10-0-129-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-152.ec2.internal' and this object" podUID="b6656694-4555-46fc-b822-5a279c511a13" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"
Jul 09 15:12:20.500453 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:20.500408    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnx2c" event={"ID":"ae42b068-625d-4729-bf26-8468ea371a8f","Type":"ContainerStarted","Data":"0417506c7335d6b89cde1fe5ebd494c6a7d4e0a3c436957ccda7c0d61b7ab9b6"}
Jul 09 15:12:20.500606 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:20.500470    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnx2c"
Jul 09 15:12:20.500606 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:20.500492    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnx2c" event={"ID":"ae42b068-625d-4729-bf26-8468ea371a8f","Type":"ContainerStarted","Data":"6b2eba9a05af70d3a5cce2c72904e8997be93047a4e1596902fd7c1a846cdd1e"}
Jul 09 15:12:20.504253 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:20.504229    2579 scope.go:117] "RemoveContainer" containerID="ac0178898ed27a9bc132bede7aa086cfc1a2e478c797853c4d609799b5e7b99c"
Jul 09 15:12:20.504535 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:12:20.504513    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac0178898ed27a9bc132bede7aa086cfc1a2e478c797853c4d609799b5e7b99c\": container with ID starting with ac0178898ed27a9bc132bede7aa086cfc1a2e478c797853c4d609799b5e7b99c not found: ID does not exist" containerID="ac0178898ed27a9bc132bede7aa086cfc1a2e478c797853c4d609799b5e7b99c"
Jul 09 15:12:20.504614 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:20.504549    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac0178898ed27a9bc132bede7aa086cfc1a2e478c797853c4d609799b5e7b99c"} err="failed to get container status \"ac0178898ed27a9bc132bede7aa086cfc1a2e478c797853c4d609799b5e7b99c\": rpc error: code = NotFound desc = could not find container \"ac0178898ed27a9bc132bede7aa086cfc1a2e478c797853c4d609799b5e7b99c\": container with ID starting with ac0178898ed27a9bc132bede7aa086cfc1a2e478c797853c4d609799b5e7b99c not found: ID does not exist"
Jul 09 15:12:20.525094 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:20.525039    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gwz7t" podStartSLOduration=2.449019766 podStartE2EDuration="25.525023811s" totalImagesPullingTime="23.076004045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:12:20.523858452 +0000 UTC m=+590.857288418" watchObservedRunningTime="2026-07-09 15:12:20.525023811 +0000 UTC m=+590.858453774"
Jul 09 15:12:20.527785 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:12:20.527755    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2q9b8\" is forbidden: User \"system:node:ip-10-0-129-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-152.ec2.internal' and this object" podUID="b6656694-4555-46fc-b822-5a279c511a13" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"
Jul 09 15:12:20.530094 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:12:20.530066    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-2q9b8\" is forbidden: User \"system:node:ip-10-0-129-152.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-152.ec2.internal' and this object" podUID="b6656694-4555-46fc-b822-5a279c511a13" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2q9b8"
Jul 09 15:12:20.557828 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:20.557770    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnx2c" podStartSLOduration=14.557753361 podStartE2EDuration="14.557753361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:12:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:12:20.55660514 +0000 UTC m=+590.890035119" watchObservedRunningTime="2026-07-09 15:12:20.557753361 +0000 UTC m=+590.891183326"
Jul 09 15:12:30.087117 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:30.087087    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:12:30.088078 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:30.088056    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:12:31.508564 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:31.508531    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hnx2c"
Jul 09 15:12:39.987734 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:39.987679    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gw24k"]
Jul 09 15:12:40.137727 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.137669    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gw24k"]
Jul 09 15:12:40.137727 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.137739    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gw24k"]
Jul 09 15:12:40.137948 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.137791    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k"
Jul 09 15:12:40.143006 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.142975    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 09 15:12:40.197879 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.197837    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q6qv\" (UniqueName: \"kubernetes.io/projected/1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354-kube-api-access-4q6qv\") pod \"limitador-limitador-64c94fd64f-gw24k\" (UID: \"1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k"
Jul 09 15:12:40.198058 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.197886    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354-config-file\") pod \"limitador-limitador-64c94fd64f-gw24k\" (UID: \"1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k"
Jul 09 15:12:40.299078 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.298985    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q6qv\" (UniqueName: \"kubernetes.io/projected/1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354-kube-api-access-4q6qv\") pod \"limitador-limitador-64c94fd64f-gw24k\" (UID: \"1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k"
Jul 09 15:12:40.299078 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.299036    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354-config-file\") pod \"limitador-limitador-64c94fd64f-gw24k\" (UID: \"1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k"
Jul 09 15:12:40.299769 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.299749    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354-config-file\") pod \"limitador-limitador-64c94fd64f-gw24k\" (UID: \"1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k"
Jul 09 15:12:40.310851 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.310814    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q6qv\" (UniqueName: \"kubernetes.io/projected/1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354-kube-api-access-4q6qv\") pod \"limitador-limitador-64c94fd64f-gw24k\" (UID: \"1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k"
Jul 09 15:12:40.447779 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.447737    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k"
Jul 09 15:12:40.464574 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.464536    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:12:40.494092 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.494007    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-74r2z"
Jul 09 15:12:40.496333 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.496306    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:12:40.523744 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.523706    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:12:40.601064 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.601025    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/563d29fd-00f4-49b4-9bbb-b6ef57f88d41-config-file\") pod \"limitador-limitador-54d6d988d5-74r2z\" (UID: \"563d29fd-00f4-49b4-9bbb-b6ef57f88d41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-74r2z"
Jul 09 15:12:40.601238 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.601072    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd6fm\" (UniqueName: \"kubernetes.io/projected/563d29fd-00f4-49b4-9bbb-b6ef57f88d41-kube-api-access-kd6fm\") pod \"limitador-limitador-54d6d988d5-74r2z\" (UID: \"563d29fd-00f4-49b4-9bbb-b6ef57f88d41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-74r2z"
Jul 09 15:12:40.611240 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.611212    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gw24k"]
Jul 09 15:12:40.620966 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:12:40.620928    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d5659d0_dc76_4f7d_8e00_e0eb9ed0f354.slice/crio-3e100843320a1d0e1b129307ba03a780c5acf2f058f2b8d8d1ea10b4a6b44cb6 WatchSource:0}: Error finding container 3e100843320a1d0e1b129307ba03a780c5acf2f058f2b8d8d1ea10b4a6b44cb6: Status 404 returned error can't find the container with id 3e100843320a1d0e1b129307ba03a780c5acf2f058f2b8d8d1ea10b4a6b44cb6
Jul 09 15:12:40.702391 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.702349    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd6fm\" (UniqueName: \"kubernetes.io/projected/563d29fd-00f4-49b4-9bbb-b6ef57f88d41-kube-api-access-kd6fm\") pod \"limitador-limitador-54d6d988d5-74r2z\" (UID: \"563d29fd-00f4-49b4-9bbb-b6ef57f88d41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-74r2z"
Jul 09 15:12:40.702644 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.702434    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/563d29fd-00f4-49b4-9bbb-b6ef57f88d41-config-file\") pod \"limitador-limitador-54d6d988d5-74r2z\" (UID: \"563d29fd-00f4-49b4-9bbb-b6ef57f88d41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-74r2z"
Jul 09 15:12:40.703097 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.703078    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/563d29fd-00f4-49b4-9bbb-b6ef57f88d41-config-file\") pod \"limitador-limitador-54d6d988d5-74r2z\" (UID: \"563d29fd-00f4-49b4-9bbb-b6ef57f88d41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-74r2z"
Jul 09 15:12:40.712729 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.712703    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd6fm\" (UniqueName: \"kubernetes.io/projected/563d29fd-00f4-49b4-9bbb-b6ef57f88d41-kube-api-access-kd6fm\") pod \"limitador-limitador-54d6d988d5-74r2z\" (UID: \"563d29fd-00f4-49b4-9bbb-b6ef57f88d41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-74r2z"
Jul 09 15:12:40.805820 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.805714    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-74r2z"
Jul 09 15:12:40.901517 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.901480    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-x5flw"]
Jul 09 15:12:40.923696 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.923650    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-x5flw"]
Jul 09 15:12:40.923847 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.923795    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-x5flw"
Jul 09 15:12:40.928523 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:40.928495    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-w9ks5\""
Jul 09 15:12:41.005249 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:41.005204    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7jsh\" (UniqueName: \"kubernetes.io/projected/6205f97a-3b5d-46cb-9b05-ce909a3591b7-kube-api-access-p7jsh\") pod \"authorino-58cd955675-x5flw\" (UID: \"6205f97a-3b5d-46cb-9b05-ce909a3591b7\") " pod="kuadrant-system/authorino-58cd955675-x5flw"
Jul 09 15:12:41.036380 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:41.036352    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:12:41.037986 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:12:41.037958    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod563d29fd_00f4_49b4_9bbb_b6ef57f88d41.slice/crio-cc50fa06ae4966225c52b1fbc8dee7bae6995720c5d8662d8dfb7504b309399e WatchSource:0}: Error finding container cc50fa06ae4966225c52b1fbc8dee7bae6995720c5d8662d8dfb7504b309399e: Status 404 returned error can't find the container with id cc50fa06ae4966225c52b1fbc8dee7bae6995720c5d8662d8dfb7504b309399e
Jul 09 15:12:41.106332 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:41.106235    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7jsh\" (UniqueName: \"kubernetes.io/projected/6205f97a-3b5d-46cb-9b05-ce909a3591b7-kube-api-access-p7jsh\") pod \"authorino-58cd955675-x5flw\" (UID: \"6205f97a-3b5d-46cb-9b05-ce909a3591b7\") " pod="kuadrant-system/authorino-58cd955675-x5flw"
Jul 09 15:12:41.118279 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:41.118248    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7jsh\" (UniqueName: \"kubernetes.io/projected/6205f97a-3b5d-46cb-9b05-ce909a3591b7-kube-api-access-p7jsh\") pod \"authorino-58cd955675-x5flw\" (UID: \"6205f97a-3b5d-46cb-9b05-ce909a3591b7\") " pod="kuadrant-system/authorino-58cd955675-x5flw"
Jul 09 15:12:41.238338 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:41.238300    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-x5flw"
Jul 09 15:12:41.411211 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:41.411129    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-x5flw"]
Jul 09 15:12:41.414381 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:12:41.414351    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6205f97a_3b5d_46cb_9b05_ce909a3591b7.slice/crio-8604e22168a0a5f221420273466479797ee2a917ea6eb4432235df5ce8489fc8 WatchSource:0}: Error finding container 8604e22168a0a5f221420273466479797ee2a917ea6eb4432235df5ce8489fc8: Status 404 returned error can't find the container with id 8604e22168a0a5f221420273466479797ee2a917ea6eb4432235df5ce8489fc8
Jul 09 15:12:41.595860 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:41.595823    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-74r2z" event={"ID":"563d29fd-00f4-49b4-9bbb-b6ef57f88d41","Type":"ContainerStarted","Data":"cc50fa06ae4966225c52b1fbc8dee7bae6995720c5d8662d8dfb7504b309399e"}
Jul 09 15:12:41.597097 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:41.597058    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k" event={"ID":"1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354","Type":"ContainerStarted","Data":"3e100843320a1d0e1b129307ba03a780c5acf2f058f2b8d8d1ea10b4a6b44cb6"}
Jul 09 15:12:41.598263 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:41.598237    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-x5flw" event={"ID":"6205f97a-3b5d-46cb-9b05-ce909a3591b7","Type":"ContainerStarted","Data":"8604e22168a0a5f221420273466479797ee2a917ea6eb4432235df5ce8489fc8"}
Jul 09 15:12:44.616374 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:44.616333    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-74r2z" event={"ID":"563d29fd-00f4-49b4-9bbb-b6ef57f88d41","Type":"ContainerStarted","Data":"66ca24e0e8a436109af9d54e5bc4f45de5fab674cf07797569b6d0328eae55b0"}
Jul 09 15:12:44.617444 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:44.616418    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-74r2z"
Jul 09 15:12:44.618314 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:44.618286    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k" event={"ID":"1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354","Type":"ContainerStarted","Data":"75b4b29edfe81de0bc83ffbabca74ae94eab32f095edcbe9da58d688713b3eeb"}
Jul 09 15:12:44.618438 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:44.618384    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k"
Jul 09 15:12:44.672206 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:44.672143    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k" podStartSLOduration=2.340552369 podStartE2EDuration="5.672119186s" totalImagesPullingTime="3.331566817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:12:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:12:44.669922241 +0000 UTC m=+615.003352215" watchObservedRunningTime="2026-07-09 15:12:44.672119186 +0000 UTC m=+615.005549155"
Jul 09 15:12:44.672363 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:44.672261    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-74r2z" podStartSLOduration=1.755942153 podStartE2EDuration="4.672253815s" totalImagesPullingTime="2.916311662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:12:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:12:44.643960532 +0000 UTC m=+614.977390497" watchObservedRunningTime="2026-07-09 15:12:44.672253815 +0000 UTC m=+615.005683785"
Jul 09 15:12:46.871296 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:46.871246    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-x5flw"]
Jul 09 15:12:47.635707 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:47.635605    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-x5flw" event={"ID":"6205f97a-3b5d-46cb-9b05-ce909a3591b7","Type":"ContainerStarted","Data":"6b565205a0e188eaaff9b5a7e92dec2d65f19c0d41be4e3960749f7a1b10b7e9"}
Jul 09 15:12:47.635707 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:47.635625    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-x5flw" podUID="6205f97a-3b5d-46cb-9b05-ce909a3591b7" containerName="authorino" containerID="cri-o://6b565205a0e188eaaff9b5a7e92dec2d65f19c0d41be4e3960749f7a1b10b7e9" gracePeriod=30
Jul 09 15:12:47.662667 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:47.662619    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-x5flw" podStartSLOduration=1.9065630850000002 podStartE2EDuration="7.662603936s" totalImagesPullingTime="5.756040851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:12:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:12:47.661999638 +0000 UTC m=+617.995429604" watchObservedRunningTime="2026-07-09 15:12:47.662603936 +0000 UTC m=+617.996033941"
Jul 09 15:12:47.929551 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:47.929514    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-x5flw"
Jul 09 15:12:48.068386 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:48.068351    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7jsh\" (UniqueName: \"kubernetes.io/projected/6205f97a-3b5d-46cb-9b05-ce909a3591b7-kube-api-access-p7jsh\") pod \"6205f97a-3b5d-46cb-9b05-ce909a3591b7\" (UID: \"6205f97a-3b5d-46cb-9b05-ce909a3591b7\") "
Jul 09 15:12:48.070418 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:48.070388    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6205f97a-3b5d-46cb-9b05-ce909a3591b7-kube-api-access-p7jsh" (OuterVolumeSpecName: "kube-api-access-p7jsh") pod "6205f97a-3b5d-46cb-9b05-ce909a3591b7" (UID: "6205f97a-3b5d-46cb-9b05-ce909a3591b7"). InnerVolumeSpecName "kube-api-access-p7jsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:12:48.169648 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:48.169609    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7jsh\" (UniqueName: \"kubernetes.io/projected/6205f97a-3b5d-46cb-9b05-ce909a3591b7-kube-api-access-p7jsh\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:12:48.640959 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:48.640877    2579 generic.go:356] "Generic (PLEG): container finished" podID="6205f97a-3b5d-46cb-9b05-ce909a3591b7" containerID="6b565205a0e188eaaff9b5a7e92dec2d65f19c0d41be4e3960749f7a1b10b7e9" exitCode=0
Jul 09 15:12:48.640959 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:48.640924    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-x5flw"
Jul 09 15:12:48.640959 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:48.640947    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-x5flw" event={"ID":"6205f97a-3b5d-46cb-9b05-ce909a3591b7","Type":"ContainerDied","Data":"6b565205a0e188eaaff9b5a7e92dec2d65f19c0d41be4e3960749f7a1b10b7e9"}
Jul 09 15:12:48.641177 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:48.640980    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-x5flw" event={"ID":"6205f97a-3b5d-46cb-9b05-ce909a3591b7","Type":"ContainerDied","Data":"8604e22168a0a5f221420273466479797ee2a917ea6eb4432235df5ce8489fc8"}
Jul 09 15:12:48.641177 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:48.640997    2579 scope.go:117] "RemoveContainer" containerID="6b565205a0e188eaaff9b5a7e92dec2d65f19c0d41be4e3960749f7a1b10b7e9"
Jul 09 15:12:48.649937 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:48.649915    2579 scope.go:117] "RemoveContainer" containerID="6b565205a0e188eaaff9b5a7e92dec2d65f19c0d41be4e3960749f7a1b10b7e9"
Jul 09 15:12:48.650178 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:12:48.650160    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b565205a0e188eaaff9b5a7e92dec2d65f19c0d41be4e3960749f7a1b10b7e9\": container with ID starting with 6b565205a0e188eaaff9b5a7e92dec2d65f19c0d41be4e3960749f7a1b10b7e9 not found: ID does not exist" containerID="6b565205a0e188eaaff9b5a7e92dec2d65f19c0d41be4e3960749f7a1b10b7e9"
Jul 09 15:12:48.650241 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:48.650190    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b565205a0e188eaaff9b5a7e92dec2d65f19c0d41be4e3960749f7a1b10b7e9"} err="failed to get container status \"6b565205a0e188eaaff9b5a7e92dec2d65f19c0d41be4e3960749f7a1b10b7e9\": rpc error: code = NotFound desc = could not find container \"6b565205a0e188eaaff9b5a7e92dec2d65f19c0d41be4e3960749f7a1b10b7e9\": container with ID starting with 6b565205a0e188eaaff9b5a7e92dec2d65f19c0d41be4e3960749f7a1b10b7e9 not found: ID does not exist"
Jul 09 15:12:48.667486 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:48.667455    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-x5flw"]
Jul 09 15:12:48.673714 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:48.673675    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-x5flw"]
Jul 09 15:12:50.230371 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:50.230339    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6205f97a-3b5d-46cb-9b05-ce909a3591b7" path="/var/lib/kubelet/pods/6205f97a-3b5d-46cb-9b05-ce909a3591b7/volumes"
Jul 09 15:12:55.624600 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:55.624524    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-74r2z"
Jul 09 15:12:55.624600 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:55.624599    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k"
Jul 09 15:12:55.724578 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:55.724546    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gw24k"]
Jul 09 15:12:55.724767 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:55.724743    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k" podUID="1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354" containerName="limitador" containerID="cri-o://75b4b29edfe81de0bc83ffbabca74ae94eab32f095edcbe9da58d688713b3eeb" gracePeriod=30
Jul 09 15:12:56.671803 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:56.671780    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k"
Jul 09 15:12:56.677031 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:56.677006    2579 generic.go:356] "Generic (PLEG): container finished" podID="1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354" containerID="75b4b29edfe81de0bc83ffbabca74ae94eab32f095edcbe9da58d688713b3eeb" exitCode=0
Jul 09 15:12:56.677156 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:56.677051    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k"
Jul 09 15:12:56.677156 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:56.677090    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k" event={"ID":"1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354","Type":"ContainerDied","Data":"75b4b29edfe81de0bc83ffbabca74ae94eab32f095edcbe9da58d688713b3eeb"}
Jul 09 15:12:56.677229 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:56.677157    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-gw24k" event={"ID":"1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354","Type":"ContainerDied","Data":"3e100843320a1d0e1b129307ba03a780c5acf2f058f2b8d8d1ea10b4a6b44cb6"}
Jul 09 15:12:56.677229 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:56.677202    2579 scope.go:117] "RemoveContainer" containerID="75b4b29edfe81de0bc83ffbabca74ae94eab32f095edcbe9da58d688713b3eeb"
Jul 09 15:12:56.685470 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:56.685453    2579 scope.go:117] "RemoveContainer" containerID="75b4b29edfe81de0bc83ffbabca74ae94eab32f095edcbe9da58d688713b3eeb"
Jul 09 15:12:56.685738 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:12:56.685713    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75b4b29edfe81de0bc83ffbabca74ae94eab32f095edcbe9da58d688713b3eeb\": container with ID starting with 75b4b29edfe81de0bc83ffbabca74ae94eab32f095edcbe9da58d688713b3eeb not found: ID does not exist" containerID="75b4b29edfe81de0bc83ffbabca74ae94eab32f095edcbe9da58d688713b3eeb"
Jul 09 15:12:56.685839 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:56.685743    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75b4b29edfe81de0bc83ffbabca74ae94eab32f095edcbe9da58d688713b3eeb"} err="failed to get container status \"75b4b29edfe81de0bc83ffbabca74ae94eab32f095edcbe9da58d688713b3eeb\": rpc error: code = NotFound desc = could not find container \"75b4b29edfe81de0bc83ffbabca74ae94eab32f095edcbe9da58d688713b3eeb\": container with ID starting with 75b4b29edfe81de0bc83ffbabca74ae94eab32f095edcbe9da58d688713b3eeb not found: ID does not exist"
Jul 09 15:12:56.736550 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:56.736516    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354-config-file\") pod \"1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354\" (UID: \"1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354\") "
Jul 09 15:12:56.736723 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:56.736574    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4q6qv\" (UniqueName: \"kubernetes.io/projected/1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354-kube-api-access-4q6qv\") pod \"1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354\" (UID: \"1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354\") "
Jul 09 15:12:56.736954 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:56.736930    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354-config-file" (OuterVolumeSpecName: "config-file") pod "1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354" (UID: "1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:12:56.738628 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:56.738598    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354-kube-api-access-4q6qv" (OuterVolumeSpecName: "kube-api-access-4q6qv") pod "1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354" (UID: "1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354"). InnerVolumeSpecName "kube-api-access-4q6qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:12:56.837215 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:56.837132    2579 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354-config-file\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:12:56.837215 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:56.837161    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q6qv\" (UniqueName: \"kubernetes.io/projected/1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354-kube-api-access-4q6qv\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:12:57.002045 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:57.002006    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gw24k"]
Jul 09 15:12:57.011723 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:57.011680    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-gw24k"]
Jul 09 15:12:58.230331 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:12:58.230296    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354" path="/var/lib/kubelet/pods/1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354/volumes"
Jul 09 15:13:14.645066 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:14.645033    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lqtkt"]
Jul 09 15:13:14.650127 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:14.645335    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6205f97a-3b5d-46cb-9b05-ce909a3591b7" containerName="authorino"
Jul 09 15:13:14.650127 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:14.645344    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6205f97a-3b5d-46cb-9b05-ce909a3591b7" containerName="authorino"
Jul 09 15:13:14.650127 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:14.645358    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354" containerName="limitador"
Jul 09 15:13:14.650127 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:14.645363    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354" containerName="limitador"
Jul 09 15:13:14.650127 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:14.645413    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d5659d0-dc76-4f7d-8e00-e0eb9ed0f354" containerName="limitador"
Jul 09 15:13:14.650127 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:14.645423    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6205f97a-3b5d-46cb-9b05-ce909a3591b7" containerName="authorino"
Jul 09 15:13:14.650532 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:14.650516    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lqtkt"
Jul 09 15:13:14.654928 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:14.654908    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-w9ks5\""
Jul 09 15:13:14.667595 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:14.667574    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lqtkt"]
Jul 09 15:13:14.776062 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:14.776021    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-757wq\" (UniqueName: \"kubernetes.io/projected/7cdaaac1-7054-46d0-b484-8ffb7fa129cd-kube-api-access-757wq\") pod \"authorino-5d4b67c446-lqtkt\" (UID: \"7cdaaac1-7054-46d0-b484-8ffb7fa129cd\") " pod="kuadrant-system/authorino-5d4b67c446-lqtkt"
Jul 09 15:13:14.876543 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:14.876513    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-757wq\" (UniqueName: \"kubernetes.io/projected/7cdaaac1-7054-46d0-b484-8ffb7fa129cd-kube-api-access-757wq\") pod \"authorino-5d4b67c446-lqtkt\" (UID: \"7cdaaac1-7054-46d0-b484-8ffb7fa129cd\") " pod="kuadrant-system/authorino-5d4b67c446-lqtkt"
Jul 09 15:13:14.923546 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:14.923521    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-757wq\" (UniqueName: \"kubernetes.io/projected/7cdaaac1-7054-46d0-b484-8ffb7fa129cd-kube-api-access-757wq\") pod \"authorino-5d4b67c446-lqtkt\" (UID: \"7cdaaac1-7054-46d0-b484-8ffb7fa129cd\") " pod="kuadrant-system/authorino-5d4b67c446-lqtkt"
Jul 09 15:13:14.951304 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:14.951273    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lqtkt"]
Jul 09 15:13:14.951525 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:14.951512    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lqtkt"
Jul 09 15:13:15.101467 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.101433    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56b7ccd574-pgvlm"]
Jul 09 15:13:15.105290 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.105273    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56b7ccd574-pgvlm"
Jul 09 15:13:15.130069 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.130049    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lqtkt"]
Jul 09 15:13:15.132253 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:13:15.132220    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7cdaaac1_7054_46d0_b484_8ffb7fa129cd.slice/crio-53719d06a7f29c0c1a53abde722e73130a877846a78dfb3a1c2fc3c7e0eada0e WatchSource:0}: Error finding container 53719d06a7f29c0c1a53abde722e73130a877846a78dfb3a1c2fc3c7e0eada0e: Status 404 returned error can't find the container with id 53719d06a7f29c0c1a53abde722e73130a877846a78dfb3a1c2fc3c7e0eada0e
Jul 09 15:13:15.133709 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.133671    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56b7ccd574-pgvlm"]
Jul 09 15:13:15.280933 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.280857    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6gft\" (UniqueName: \"kubernetes.io/projected/d9a657a9-3299-4eed-99fd-d6a3f60e638d-kube-api-access-s6gft\") pod \"authorino-56b7ccd574-pgvlm\" (UID: \"d9a657a9-3299-4eed-99fd-d6a3f60e638d\") " pod="kuadrant-system/authorino-56b7ccd574-pgvlm"
Jul 09 15:13:15.381522 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.381484    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6gft\" (UniqueName: \"kubernetes.io/projected/d9a657a9-3299-4eed-99fd-d6a3f60e638d-kube-api-access-s6gft\") pod \"authorino-56b7ccd574-pgvlm\" (UID: \"d9a657a9-3299-4eed-99fd-d6a3f60e638d\") " pod="kuadrant-system/authorino-56b7ccd574-pgvlm"
Jul 09 15:13:15.390518 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.390491    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56b7ccd574-pgvlm"]
Jul 09 15:13:15.390776 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:13:15.390754    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-s6gft], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-56b7ccd574-pgvlm" podUID="d9a657a9-3299-4eed-99fd-d6a3f60e638d"
Jul 09 15:13:15.415080 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.415046    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6gft\" (UniqueName: \"kubernetes.io/projected/d9a657a9-3299-4eed-99fd-d6a3f60e638d-kube-api-access-s6gft\") pod \"authorino-56b7ccd574-pgvlm\" (UID: \"d9a657a9-3299-4eed-99fd-d6a3f60e638d\") " pod="kuadrant-system/authorino-56b7ccd574-pgvlm"
Jul 09 15:13:15.465759 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.465724    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-676cdbf88b-njrrn"]
Jul 09 15:13:15.469175 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.469153    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-676cdbf88b-njrrn"
Jul 09 15:13:15.477260 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.477230    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 09 15:13:15.499252 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.499230    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-676cdbf88b-njrrn"]
Jul 09 15:13:15.583444 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.583357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6hrc\" (UniqueName: \"kubernetes.io/projected/8299963e-2655-4b33-b866-171695e01e19-kube-api-access-j6hrc\") pod \"authorino-676cdbf88b-njrrn\" (UID: \"8299963e-2655-4b33-b866-171695e01e19\") " pod="kuadrant-system/authorino-676cdbf88b-njrrn"
Jul 09 15:13:15.583444 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.583396    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8299963e-2655-4b33-b866-171695e01e19-tls-cert\") pod \"authorino-676cdbf88b-njrrn\" (UID: \"8299963e-2655-4b33-b866-171695e01e19\") " pod="kuadrant-system/authorino-676cdbf88b-njrrn"
Jul 09 15:13:15.684873 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.684833    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6hrc\" (UniqueName: \"kubernetes.io/projected/8299963e-2655-4b33-b866-171695e01e19-kube-api-access-j6hrc\") pod \"authorino-676cdbf88b-njrrn\" (UID: \"8299963e-2655-4b33-b866-171695e01e19\") " pod="kuadrant-system/authorino-676cdbf88b-njrrn"
Jul 09 15:13:15.684873 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.684873    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8299963e-2655-4b33-b866-171695e01e19-tls-cert\") pod \"authorino-676cdbf88b-njrrn\" (UID: \"8299963e-2655-4b33-b866-171695e01e19\") " pod="kuadrant-system/authorino-676cdbf88b-njrrn"
Jul 09 15:13:15.687218 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.687195    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8299963e-2655-4b33-b866-171695e01e19-tls-cert\") pod \"authorino-676cdbf88b-njrrn\" (UID: \"8299963e-2655-4b33-b866-171695e01e19\") " pod="kuadrant-system/authorino-676cdbf88b-njrrn"
Jul 09 15:13:15.715404 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.715371    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6hrc\" (UniqueName: \"kubernetes.io/projected/8299963e-2655-4b33-b866-171695e01e19-kube-api-access-j6hrc\") pod \"authorino-676cdbf88b-njrrn\" (UID: \"8299963e-2655-4b33-b866-171695e01e19\") " pod="kuadrant-system/authorino-676cdbf88b-njrrn"
Jul 09 15:13:15.770881 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.770844    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lqtkt" event={"ID":"7cdaaac1-7054-46d0-b484-8ffb7fa129cd","Type":"ContainerStarted","Data":"f132c6868d4a8ac4190468a0db013a3b4885bfcd6fd26072c02ef4f9a9f0b7fe"}
Jul 09 15:13:15.771078 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.770883    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-lqtkt" podUID="7cdaaac1-7054-46d0-b484-8ffb7fa129cd" containerName="authorino" containerID="cri-o://f132c6868d4a8ac4190468a0db013a3b4885bfcd6fd26072c02ef4f9a9f0b7fe" gracePeriod=30
Jul 09 15:13:15.771078 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.770891    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lqtkt" event={"ID":"7cdaaac1-7054-46d0-b484-8ffb7fa129cd","Type":"ContainerStarted","Data":"53719d06a7f29c0c1a53abde722e73130a877846a78dfb3a1c2fc3c7e0eada0e"}
Jul 09 15:13:15.771078 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.770858    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56b7ccd574-pgvlm"
Jul 09 15:13:15.778372 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.778346    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56b7ccd574-pgvlm"
Jul 09 15:13:15.780912 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.780893    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-676cdbf88b-njrrn"
Jul 09 15:13:15.886188 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.886163    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6gft\" (UniqueName: \"kubernetes.io/projected/d9a657a9-3299-4eed-99fd-d6a3f60e638d-kube-api-access-s6gft\") pod \"d9a657a9-3299-4eed-99fd-d6a3f60e638d\" (UID: \"d9a657a9-3299-4eed-99fd-d6a3f60e638d\") "
Jul 09 15:13:15.888262 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.888232    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a657a9-3299-4eed-99fd-d6a3f60e638d-kube-api-access-s6gft" (OuterVolumeSpecName: "kube-api-access-s6gft") pod "d9a657a9-3299-4eed-99fd-d6a3f60e638d" (UID: "d9a657a9-3299-4eed-99fd-d6a3f60e638d"). InnerVolumeSpecName "kube-api-access-s6gft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:13:15.897829 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.897774    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-lqtkt" podStartSLOduration=1.552645624 podStartE2EDuration="1.897758154s" totalImagesPullingTime="345.11253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:13:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:13:15.828924594 +0000 UTC m=+646.162354563" watchObservedRunningTime="2026-07-09 15:13:15.897758154 +0000 UTC m=+646.231188119"
Jul 09 15:13:15.987369 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:15.987337    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6gft\" (UniqueName: \"kubernetes.io/projected/d9a657a9-3299-4eed-99fd-d6a3f60e638d-kube-api-access-s6gft\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:13:16.016595 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.016570    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lqtkt"
Jul 09 15:13:16.060697 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:13:16.060644    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8299963e_2655_4b33_b866_171695e01e19.slice/crio-efb2fae276832b4331fabb87e0735954d602bfd60d5f10cc05cddca286199e28 WatchSource:0}: Error finding container efb2fae276832b4331fabb87e0735954d602bfd60d5f10cc05cddca286199e28: Status 404 returned error can't find the container with id efb2fae276832b4331fabb87e0735954d602bfd60d5f10cc05cddca286199e28
Jul 09 15:13:16.073760 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.073732    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-676cdbf88b-njrrn"]
Jul 09 15:13:16.189499 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.189468    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-757wq\" (UniqueName: \"kubernetes.io/projected/7cdaaac1-7054-46d0-b484-8ffb7fa129cd-kube-api-access-757wq\") pod \"7cdaaac1-7054-46d0-b484-8ffb7fa129cd\" (UID: \"7cdaaac1-7054-46d0-b484-8ffb7fa129cd\") "
Jul 09 15:13:16.191404 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.191381    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cdaaac1-7054-46d0-b484-8ffb7fa129cd-kube-api-access-757wq" (OuterVolumeSpecName: "kube-api-access-757wq") pod "7cdaaac1-7054-46d0-b484-8ffb7fa129cd" (UID: "7cdaaac1-7054-46d0-b484-8ffb7fa129cd"). InnerVolumeSpecName "kube-api-access-757wq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:13:16.290070 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.290038    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-757wq\" (UniqueName: \"kubernetes.io/projected/7cdaaac1-7054-46d0-b484-8ffb7fa129cd-kube-api-access-757wq\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:13:16.775967 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.775874    2579 generic.go:356] "Generic (PLEG): container finished" podID="7cdaaac1-7054-46d0-b484-8ffb7fa129cd" containerID="f132c6868d4a8ac4190468a0db013a3b4885bfcd6fd26072c02ef4f9a9f0b7fe" exitCode=0
Jul 09 15:13:16.775967 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.775921    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-lqtkt"
Jul 09 15:13:16.775967 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.775934    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lqtkt" event={"ID":"7cdaaac1-7054-46d0-b484-8ffb7fa129cd","Type":"ContainerDied","Data":"f132c6868d4a8ac4190468a0db013a3b4885bfcd6fd26072c02ef4f9a9f0b7fe"}
Jul 09 15:13:16.776490 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.775981    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-lqtkt" event={"ID":"7cdaaac1-7054-46d0-b484-8ffb7fa129cd","Type":"ContainerDied","Data":"53719d06a7f29c0c1a53abde722e73130a877846a78dfb3a1c2fc3c7e0eada0e"}
Jul 09 15:13:16.776490 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.775999    2579 scope.go:117] "RemoveContainer" containerID="f132c6868d4a8ac4190468a0db013a3b4885bfcd6fd26072c02ef4f9a9f0b7fe"
Jul 09 15:13:16.777822 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.777766    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-676cdbf88b-njrrn" event={"ID":"8299963e-2655-4b33-b866-171695e01e19","Type":"ContainerStarted","Data":"f7c7b0ebdd580e4a28caaa6f3be278bea23616389c24cdd67db1af06187b0051"}
Jul 09 15:13:16.777822 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.777797    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56b7ccd574-pgvlm"
Jul 09 15:13:16.777822 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.777811    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-676cdbf88b-njrrn" event={"ID":"8299963e-2655-4b33-b866-171695e01e19","Type":"ContainerStarted","Data":"efb2fae276832b4331fabb87e0735954d602bfd60d5f10cc05cddca286199e28"}
Jul 09 15:13:16.784571 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.784542    2579 scope.go:117] "RemoveContainer" containerID="f132c6868d4a8ac4190468a0db013a3b4885bfcd6fd26072c02ef4f9a9f0b7fe"
Jul 09 15:13:16.784868 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:13:16.784851    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f132c6868d4a8ac4190468a0db013a3b4885bfcd6fd26072c02ef4f9a9f0b7fe\": container with ID starting with f132c6868d4a8ac4190468a0db013a3b4885bfcd6fd26072c02ef4f9a9f0b7fe not found: ID does not exist" containerID="f132c6868d4a8ac4190468a0db013a3b4885bfcd6fd26072c02ef4f9a9f0b7fe"
Jul 09 15:13:16.784933 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.784880    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f132c6868d4a8ac4190468a0db013a3b4885bfcd6fd26072c02ef4f9a9f0b7fe"} err="failed to get container status \"f132c6868d4a8ac4190468a0db013a3b4885bfcd6fd26072c02ef4f9a9f0b7fe\": rpc error: code = NotFound desc = could not find container \"f132c6868d4a8ac4190468a0db013a3b4885bfcd6fd26072c02ef4f9a9f0b7fe\": container with ID starting with f132c6868d4a8ac4190468a0db013a3b4885bfcd6fd26072c02ef4f9a9f0b7fe not found: ID does not exist"
Jul 09 15:13:16.844069 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.844035    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56b7ccd574-pgvlm"]
Jul 09 15:13:16.853656 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.853623    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56b7ccd574-pgvlm"]
Jul 09 15:13:16.902225 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.902173    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-676cdbf88b-njrrn" podStartSLOduration=1.4441842999999999 podStartE2EDuration="1.902155792s" totalImagesPullingTime="457.971492ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:13:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:13:16.899077671 +0000 UTC m=+647.232507648" watchObservedRunningTime="2026-07-09 15:13:16.902155792 +0000 UTC m=+647.235585757"
Jul 09 15:13:16.950721 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.950663    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lqtkt"]
Jul 09 15:13:16.982610 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:16.982576    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-lqtkt"]
Jul 09 15:13:18.230308 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:18.230275    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cdaaac1-7054-46d0-b484-8ffb7fa129cd" path="/var/lib/kubelet/pods/7cdaaac1-7054-46d0-b484-8ffb7fa129cd/volumes"
Jul 09 15:13:18.230700 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:18.230572    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9a657a9-3299-4eed-99fd-d6a3f60e638d" path="/var/lib/kubelet/pods/d9a657a9-3299-4eed-99fd-d6a3f60e638d/volumes"
Jul 09 15:13:59.136294 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.136255    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"]
Jul 09 15:13:59.136743 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.136551    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cdaaac1-7054-46d0-b484-8ffb7fa129cd" containerName="authorino"
Jul 09 15:13:59.136743 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.136562    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cdaaac1-7054-46d0-b484-8ffb7fa129cd" containerName="authorino"
Jul 09 15:13:59.136743 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.136634    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cdaaac1-7054-46d0-b484-8ffb7fa129cd" containerName="authorino"
Jul 09 15:13:59.138609 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.138594    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"
Jul 09 15:13:59.141517 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.141495    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 09 15:13:59.141673 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.141498    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 15:13:59.141870 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.141855    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9r9rl\""
Jul 09 15:13:59.142745 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.142723    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 15:13:59.152639 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.152606    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"]
Jul 09 15:13:59.216114 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.216074    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ab3744c-1e33-480c-bf7c-afe308d5588a-openshift-service-ca\") pod \"maas-api-5f8f8f9c45-rs4zn\" (UID: \"7ab3744c-1e33-480c-bf7c-afe308d5588a\") " pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"
Jul 09 15:13:59.216296 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.216119    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxx2f\" (UniqueName: \"kubernetes.io/projected/7ab3744c-1e33-480c-bf7c-afe308d5588a-kube-api-access-qxx2f\") pod \"maas-api-5f8f8f9c45-rs4zn\" (UID: \"7ab3744c-1e33-480c-bf7c-afe308d5588a\") " pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"
Jul 09 15:13:59.216296 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.216207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ab3744c-1e33-480c-bf7c-afe308d5588a-maas-api-tls\") pod \"maas-api-5f8f8f9c45-rs4zn\" (UID: \"7ab3744c-1e33-480c-bf7c-afe308d5588a\") " pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"
Jul 09 15:13:59.317300 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.317257    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ab3744c-1e33-480c-bf7c-afe308d5588a-openshift-service-ca\") pod \"maas-api-5f8f8f9c45-rs4zn\" (UID: \"7ab3744c-1e33-480c-bf7c-afe308d5588a\") " pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"
Jul 09 15:13:59.317300 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.317304    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxx2f\" (UniqueName: \"kubernetes.io/projected/7ab3744c-1e33-480c-bf7c-afe308d5588a-kube-api-access-qxx2f\") pod \"maas-api-5f8f8f9c45-rs4zn\" (UID: \"7ab3744c-1e33-480c-bf7c-afe308d5588a\") " pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"
Jul 09 15:13:59.317509 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.317339    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ab3744c-1e33-480c-bf7c-afe308d5588a-maas-api-tls\") pod \"maas-api-5f8f8f9c45-rs4zn\" (UID: \"7ab3744c-1e33-480c-bf7c-afe308d5588a\") " pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"
Jul 09 15:13:59.317911 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.317880    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ab3744c-1e33-480c-bf7c-afe308d5588a-openshift-service-ca\") pod \"maas-api-5f8f8f9c45-rs4zn\" (UID: \"7ab3744c-1e33-480c-bf7c-afe308d5588a\") " pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"
Jul 09 15:13:59.319750 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.319729    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ab3744c-1e33-480c-bf7c-afe308d5588a-maas-api-tls\") pod \"maas-api-5f8f8f9c45-rs4zn\" (UID: \"7ab3744c-1e33-480c-bf7c-afe308d5588a\") " pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"
Jul 09 15:13:59.337955 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.337931    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxx2f\" (UniqueName: \"kubernetes.io/projected/7ab3744c-1e33-480c-bf7c-afe308d5588a-kube-api-access-qxx2f\") pod \"maas-api-5f8f8f9c45-rs4zn\" (UID: \"7ab3744c-1e33-480c-bf7c-afe308d5588a\") " pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"
Jul 09 15:13:59.448379 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.448346    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"
Jul 09 15:13:59.599648 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.599624    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"]
Jul 09 15:13:59.601357 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:13:59.601330    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ab3744c_1e33_480c_bf7c_afe308d5588a.slice/crio-19393fb8020bf4ce474547febcf6d73d9185c0a5737847d0bb726cfd784f7cea WatchSource:0}: Error finding container 19393fb8020bf4ce474547febcf6d73d9185c0a5737847d0bb726cfd784f7cea: Status 404 returned error can't find the container with id 19393fb8020bf4ce474547febcf6d73d9185c0a5737847d0bb726cfd784f7cea
Jul 09 15:13:59.972997 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:13:59.972965    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn" event={"ID":"7ab3744c-1e33-480c-bf7c-afe308d5588a","Type":"ContainerStarted","Data":"19393fb8020bf4ce474547febcf6d73d9185c0a5737847d0bb726cfd784f7cea"}
Jul 09 15:14:00.235423 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:00.235335    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-qhk65"]
Jul 09 15:14:00.237848 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:00.237823    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-qhk65"
Jul 09 15:14:00.240844 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:00.240820    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 09 15:14:00.241452 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:00.241391    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-cbkll\""
Jul 09 15:14:00.254671 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:00.254637    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-qhk65"]
Jul 09 15:14:00.316718 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:00.316008    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:14:00.326430 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:00.326398    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsqm7\" (UniqueName: \"kubernetes.io/projected/faa2e895-99fe-471d-a028-17843034d623-kube-api-access-lsqm7\") pod \"payload-pre-processing-7964457696-qhk65\" (UID: \"faa2e895-99fe-471d-a028-17843034d623\") " pod="openshift-ingress/payload-pre-processing-7964457696-qhk65"
Jul 09 15:14:00.326642 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:00.326448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faa2e895-99fe-471d-a028-17843034d623-plugins-config-volume\") pod \"payload-pre-processing-7964457696-qhk65\" (UID: \"faa2e895-99fe-471d-a028-17843034d623\") " pod="openshift-ingress/payload-pre-processing-7964457696-qhk65"
Jul 09 15:14:00.427110 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:00.427066    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faa2e895-99fe-471d-a028-17843034d623-plugins-config-volume\") pod \"payload-pre-processing-7964457696-qhk65\" (UID: \"faa2e895-99fe-471d-a028-17843034d623\") " pod="openshift-ingress/payload-pre-processing-7964457696-qhk65"
Jul 09 15:14:00.427276 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:00.427211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsqm7\" (UniqueName: \"kubernetes.io/projected/faa2e895-99fe-471d-a028-17843034d623-kube-api-access-lsqm7\") pod \"payload-pre-processing-7964457696-qhk65\" (UID: \"faa2e895-99fe-471d-a028-17843034d623\") " pod="openshift-ingress/payload-pre-processing-7964457696-qhk65"
Jul 09 15:14:00.427946 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:00.427922    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faa2e895-99fe-471d-a028-17843034d623-plugins-config-volume\") pod \"payload-pre-processing-7964457696-qhk65\" (UID: \"faa2e895-99fe-471d-a028-17843034d623\") " pod="openshift-ingress/payload-pre-processing-7964457696-qhk65"
Jul 09 15:14:00.437718 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:00.437652    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsqm7\" (UniqueName: \"kubernetes.io/projected/faa2e895-99fe-471d-a028-17843034d623-kube-api-access-lsqm7\") pod \"payload-pre-processing-7964457696-qhk65\" (UID: \"faa2e895-99fe-471d-a028-17843034d623\") " pod="openshift-ingress/payload-pre-processing-7964457696-qhk65"
Jul 09 15:14:00.551200 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:00.551096    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-qhk65"
Jul 09 15:14:00.708932 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:00.708888    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-qhk65"]
Jul 09 15:14:00.711711 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:14:00.711642    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa2e895_99fe_471d_a028_17843034d623.slice/crio-e52b7091c939bacd844a026c2de4af6de10868a6cf65ce446f9930c3ba9977e9 WatchSource:0}: Error finding container e52b7091c939bacd844a026c2de4af6de10868a6cf65ce446f9930c3ba9977e9: Status 404 returned error can't find the container with id e52b7091c939bacd844a026c2de4af6de10868a6cf65ce446f9930c3ba9977e9
Jul 09 15:14:00.981221 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:00.981123    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-qhk65" event={"ID":"faa2e895-99fe-471d-a028-17843034d623","Type":"ContainerStarted","Data":"e52b7091c939bacd844a026c2de4af6de10868a6cf65ce446f9930c3ba9977e9"}
Jul 09 15:14:01.988658 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:01.988622    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn" event={"ID":"7ab3744c-1e33-480c-bf7c-afe308d5588a","Type":"ContainerStarted","Data":"e90ef99f087190ddcd7ac72b367c4a8ed12a044faad9d004188074fa6d4f2eb0"}
Jul 09 15:14:01.989137 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:01.988759    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"
Jul 09 15:14:02.016948 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:02.016882    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn" podStartSLOduration=0.750446714 podStartE2EDuration="3.016864672s" totalImagesPullingTime="2.266417958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:13:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:14:02.015475587 +0000 UTC m=+692.348905550" watchObservedRunningTime="2026-07-09 15:14:02.016864672 +0000 UTC m=+692.350294635"
Jul 09 15:14:02.994474 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:02.994437    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-qhk65" event={"ID":"faa2e895-99fe-471d-a028-17843034d623","Type":"ContainerStarted","Data":"73be9470354ce500d8381d2d34f56b5cce082e150c383acbcf2915a47fdfb377"}
Jul 09 15:14:02.994902 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:02.994593    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-qhk65"
Jul 09 15:14:03.016371 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:03.016319    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-qhk65" podStartSLOduration=1.169645655 podStartE2EDuration="3.016300922s" totalImagesPullingTime="1.846655267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:14:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:14:03.014851365 +0000 UTC m=+693.348281330" watchObservedRunningTime="2026-07-09 15:14:03.016300922 +0000 UTC m=+693.349730885"
Jul 09 15:14:07.999601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:07.999566    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5f8f8f9c45-rs4zn"
Jul 09 15:14:14.001273 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:14.001221    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-7964457696-qhk65" podUID="faa2e895-99fe-471d-a028-17843034d623" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.37:9004: connect: connection refused"
Jul 09 15:14:23.999466 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:23.999384    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-qhk65"
Jul 09 15:14:24.966584 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:24.966551    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:14:49.082989 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.082952    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"]
Jul 09 15:14:49.093179 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.093133    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.096173 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.096151    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 09 15:14:49.096747 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.096723    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"]
Jul 09 15:14:49.097221 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.097202    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 09 15:14:49.097316 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.097227    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 09 15:14:49.097316 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.097261    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f2vzq\""
Jul 09 15:14:49.127747 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.127702    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3437421b-7a2d-4ddb-ba57-d3cff910c40c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.127917 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.127760    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3437421b-7a2d-4ddb-ba57-d3cff910c40c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.127917 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.127779    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3437421b-7a2d-4ddb-ba57-d3cff910c40c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.127917 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.127803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3437421b-7a2d-4ddb-ba57-d3cff910c40c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.127917 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.127833    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp9n4\" (UniqueName: \"kubernetes.io/projected/3437421b-7a2d-4ddb-ba57-d3cff910c40c-kube-api-access-mp9n4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.128050 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.127940    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3437421b-7a2d-4ddb-ba57-d3cff910c40c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.229249 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.229216    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3437421b-7a2d-4ddb-ba57-d3cff910c40c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.229249 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.229254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp9n4\" (UniqueName: \"kubernetes.io/projected/3437421b-7a2d-4ddb-ba57-d3cff910c40c-kube-api-access-mp9n4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.229437 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.229372    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3437421b-7a2d-4ddb-ba57-d3cff910c40c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.229437 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.229413    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3437421b-7a2d-4ddb-ba57-d3cff910c40c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.229511 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.229437    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3437421b-7a2d-4ddb-ba57-d3cff910c40c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.229511 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.229460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3437421b-7a2d-4ddb-ba57-d3cff910c40c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.229664 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.229646    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3437421b-7a2d-4ddb-ba57-d3cff910c40c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.229724 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.229713    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3437421b-7a2d-4ddb-ba57-d3cff910c40c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.229856 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.229837    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3437421b-7a2d-4ddb-ba57-d3cff910c40c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.231478 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.231453    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3437421b-7a2d-4ddb-ba57-d3cff910c40c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.231903 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.231883    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3437421b-7a2d-4ddb-ba57-d3cff910c40c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.240873 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.240850    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp9n4\" (UniqueName: \"kubernetes.io/projected/3437421b-7a2d-4ddb-ba57-d3cff910c40c-kube-api-access-mp9n4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-s7pwb\" (UID: \"3437421b-7a2d-4ddb-ba57-d3cff910c40c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.404948 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.404844    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:49.542109 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.542084    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"]
Jul 09 15:14:49.544013 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:14:49.543991    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3437421b_7a2d_4ddb_ba57_d3cff910c40c.slice/crio-cd04054c737bf75af6e075be524f2ceaaad0b34635b90ca2b335f32039901010 WatchSource:0}: Error finding container cd04054c737bf75af6e075be524f2ceaaad0b34635b90ca2b335f32039901010: Status 404 returned error can't find the container with id cd04054c737bf75af6e075be524f2ceaaad0b34635b90ca2b335f32039901010
Jul 09 15:14:49.545261 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.545246    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 15:14:49.782628 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:49.782581    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:14:50.215373 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:50.215329    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb" event={"ID":"3437421b-7a2d-4ddb-ba57-d3cff910c40c","Type":"ContainerStarted","Data":"cd04054c737bf75af6e075be524f2ceaaad0b34635b90ca2b335f32039901010"}
Jul 09 15:14:51.221199 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:51.221160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb" event={"ID":"3437421b-7a2d-4ddb-ba57-d3cff910c40c","Type":"ContainerStarted","Data":"10e12d6d4f9b3fb8bf49619c22c3d80bec421b58ffb60096d03bcee8f6767f39"}
Jul 09 15:14:51.240931 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:51.240878    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb" podStartSLOduration=1.156205071 podStartE2EDuration="2.240860275s" totalImagesPullingTime="1.084655204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:14:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:14:51.238613023 +0000 UTC m=+741.572042986" watchObservedRunningTime="2026-07-09 15:14:51.240860275 +0000 UTC m=+741.574290243"
Jul 09 15:14:59.261292 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:59.261260    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:14:59.405965 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:59.405922    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:59.406147 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:59.405991    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:14:59.421068 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:14:59.421029    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:15:00.145331 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:00.145296    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw"]
Jul 09 15:15:00.147873 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:00.147857    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw"
Jul 09 15:15:00.157666 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:00.157641    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw"]
Jul 09 15:15:00.227406 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:00.227363    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwvh6\" (UniqueName: \"kubernetes.io/projected/f2a2887f-fc1b-4a8c-bfce-957dfc232998-kube-api-access-rwvh6\") pod \"maas-api-key-cleanup-29726835-gcxtw\" (UID: \"f2a2887f-fc1b-4a8c-bfce-957dfc232998\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw"
Jul 09 15:15:00.282676 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:00.282649    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-s7pwb"
Jul 09 15:15:00.328373 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:00.328334    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwvh6\" (UniqueName: \"kubernetes.io/projected/f2a2887f-fc1b-4a8c-bfce-957dfc232998-kube-api-access-rwvh6\") pod \"maas-api-key-cleanup-29726835-gcxtw\" (UID: \"f2a2887f-fc1b-4a8c-bfce-957dfc232998\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw"
Jul 09 15:15:00.344982 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:00.344947    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwvh6\" (UniqueName: \"kubernetes.io/projected/f2a2887f-fc1b-4a8c-bfce-957dfc232998-kube-api-access-rwvh6\") pod \"maas-api-key-cleanup-29726835-gcxtw\" (UID: \"f2a2887f-fc1b-4a8c-bfce-957dfc232998\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw"
Jul 09 15:15:00.458369 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:00.458335    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw"
Jul 09 15:15:00.588071 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:00.588048    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw"]
Jul 09 15:15:00.590175 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:15:00.590145    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a2887f_fc1b_4a8c_bfce_957dfc232998.slice/crio-b9b121795334f87c6440bf3310dc7a561f3490d828ecd992d25c3238a778548f WatchSource:0}: Error finding container b9b121795334f87c6440bf3310dc7a561f3490d828ecd992d25c3238a778548f: Status 404 returned error can't find the container with id b9b121795334f87c6440bf3310dc7a561f3490d828ecd992d25c3238a778548f
Jul 09 15:15:01.275580 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:01.275544    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw" event={"ID":"f2a2887f-fc1b-4a8c-bfce-957dfc232998","Type":"ContainerStarted","Data":"b9b121795334f87c6440bf3310dc7a561f3490d828ecd992d25c3238a778548f"}
Jul 09 15:15:02.391715 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.391654    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"]
Jul 09 15:15:02.394234 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.394211    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.397372 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.397351    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 09 15:15:02.420991 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.420959    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"]
Jul 09 15:15:02.448341 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.448311    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf2d6765-3d89-4a85-9770-1ed221fde97a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.448516 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.448364    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf2d6765-3d89-4a85-9770-1ed221fde97a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.448516 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.448433    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf2d6765-3d89-4a85-9770-1ed221fde97a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.448516 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.448474    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf2d6765-3d89-4a85-9770-1ed221fde97a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.448516 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.448509    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf2d6765-3d89-4a85-9770-1ed221fde97a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.448705 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.448547    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49xtw\" (UniqueName: \"kubernetes.io/projected/cf2d6765-3d89-4a85-9770-1ed221fde97a-kube-api-access-49xtw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.549647 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.549615    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf2d6765-3d89-4a85-9770-1ed221fde97a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.549851 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.549654    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf2d6765-3d89-4a85-9770-1ed221fde97a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.549851 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.549673    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf2d6765-3d89-4a85-9770-1ed221fde97a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.549851 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.549721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf2d6765-3d89-4a85-9770-1ed221fde97a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.549851 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.549749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49xtw\" (UniqueName: \"kubernetes.io/projected/cf2d6765-3d89-4a85-9770-1ed221fde97a-kube-api-access-49xtw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.549851 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.549804    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf2d6765-3d89-4a85-9770-1ed221fde97a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.550129 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.550109    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cf2d6765-3d89-4a85-9770-1ed221fde97a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.550253 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.550236    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf2d6765-3d89-4a85-9770-1ed221fde97a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.550321 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.550254    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cf2d6765-3d89-4a85-9770-1ed221fde97a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.552075 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.552053    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cf2d6765-3d89-4a85-9770-1ed221fde97a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.552445 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.552424    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cf2d6765-3d89-4a85-9770-1ed221fde97a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.561133 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.561109    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49xtw\" (UniqueName: \"kubernetes.io/projected/cf2d6765-3d89-4a85-9770-1ed221fde97a-kube-api-access-49xtw\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j\" (UID: \"cf2d6765-3d89-4a85-9770-1ed221fde97a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.707858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.707818    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:02.885402 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:02.885225    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"]
Jul 09 15:15:03.477407 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:15:03.477365    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf2d6765_3d89_4a85_9770_1ed221fde97a.slice/crio-2c3560491fbdf284035f44295155e52ae7a7cdc6cb94da79ade649bd0a18540c WatchSource:0}: Error finding container 2c3560491fbdf284035f44295155e52ae7a7cdc6cb94da79ade649bd0a18540c: Status 404 returned error can't find the container with id 2c3560491fbdf284035f44295155e52ae7a7cdc6cb94da79ade649bd0a18540c
Jul 09 15:15:03.701932 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:03.701890    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:15:04.292835 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:04.292798    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j" event={"ID":"cf2d6765-3d89-4a85-9770-1ed221fde97a","Type":"ContainerStarted","Data":"633bf4d1dd185298603be3215f65e03339ff506af51677b6b9520b19e09242b5"}
Jul 09 15:15:04.293030 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:04.292853    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j" event={"ID":"cf2d6765-3d89-4a85-9770-1ed221fde97a","Type":"ContainerStarted","Data":"2c3560491fbdf284035f44295155e52ae7a7cdc6cb94da79ade649bd0a18540c"}
Jul 09 15:15:04.294295 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:04.294269    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw" event={"ID":"f2a2887f-fc1b-4a8c-bfce-957dfc232998","Type":"ContainerStarted","Data":"21eb8b7381312493f0f5e7355e2dbc1111e55b52b97f025268380f9811b824aa"}
Jul 09 15:15:04.322971 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:04.322919    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j" podStartSLOduration=2.028536798 podStartE2EDuration="2.322903084s" totalImagesPullingTime="294.366286ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:15:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:15:04.321660585 +0000 UTC m=+754.655090548" watchObservedRunningTime="2026-07-09 15:15:04.322903084 +0000 UTC m=+754.656333074"
Jul 09 15:15:04.356374 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:04.356319    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw" podStartSLOduration=1.4365279389999999 podStartE2EDuration="4.356302467s" totalImagesPullingTime="2.919774528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:15:04.354972813 +0000 UTC m=+754.688402777" watchObservedRunningTime="2026-07-09 15:15:04.356302467 +0000 UTC m=+754.689732468"
Jul 09 15:15:12.708875 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:12.708830    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:12.709288 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:12.708910    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:12.721030 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:12.721003    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:13.346362 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:13.346333    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j"
Jul 09 15:15:17.457452 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:17.457413    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:15:29.661432 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.661397    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"]
Jul 09 15:15:29.664500 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.664482    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.667056 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.667037    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 09 15:15:29.677610 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.677586    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"]
Jul 09 15:15:29.777012 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.776973    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13834160-216c-420e-ad87-5b9a1a780b03-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.777196 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.777041    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13834160-216c-420e-ad87-5b9a1a780b03-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.777196 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.777100    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggzwp\" (UniqueName: \"kubernetes.io/projected/13834160-216c-420e-ad87-5b9a1a780b03-kube-api-access-ggzwp\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.777196 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.777144    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13834160-216c-420e-ad87-5b9a1a780b03-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.777196 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.777170    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13834160-216c-420e-ad87-5b9a1a780b03-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.777350 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.777231    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13834160-216c-420e-ad87-5b9a1a780b03-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.878534 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.878496    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13834160-216c-420e-ad87-5b9a1a780b03-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.878729 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.878579    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13834160-216c-420e-ad87-5b9a1a780b03-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.878729 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.878605    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggzwp\" (UniqueName: \"kubernetes.io/projected/13834160-216c-420e-ad87-5b9a1a780b03-kube-api-access-ggzwp\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.878729 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.878623    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13834160-216c-420e-ad87-5b9a1a780b03-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.878729 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.878643    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13834160-216c-420e-ad87-5b9a1a780b03-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.878729 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.878717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13834160-216c-420e-ad87-5b9a1a780b03-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.879020 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.878995    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13834160-216c-420e-ad87-5b9a1a780b03-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.879078 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.879021    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13834160-216c-420e-ad87-5b9a1a780b03-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.879124 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.879073    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13834160-216c-420e-ad87-5b9a1a780b03-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.880861 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.880839    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13834160-216c-420e-ad87-5b9a1a780b03-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.881103 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.881086    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13834160-216c-420e-ad87-5b9a1a780b03-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.887622 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.887597    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggzwp\" (UniqueName: \"kubernetes.io/projected/13834160-216c-420e-ad87-5b9a1a780b03-kube-api-access-ggzwp\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk\" (UID: \"13834160-216c-420e-ad87-5b9a1a780b03\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:29.974065 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:29.974020    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:30.114188 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:30.114149    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"]
Jul 09 15:15:30.116497 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:15:30.116460    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13834160_216c_420e_ad87_5b9a1a780b03.slice/crio-4410fc513a5b5d51ae0ad1ad07e91c9c565207e896946af3dde1838dc98bff37 WatchSource:0}: Error finding container 4410fc513a5b5d51ae0ad1ad07e91c9c565207e896946af3dde1838dc98bff37: Status 404 returned error can't find the container with id 4410fc513a5b5d51ae0ad1ad07e91c9c565207e896946af3dde1838dc98bff37
Jul 09 15:15:30.422368 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:30.422334    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk" event={"ID":"13834160-216c-420e-ad87-5b9a1a780b03","Type":"ContainerStarted","Data":"f3800292c87dbe9a14fd63af854edc55db87d756b11d112637533152ccde21c5"}
Jul 09 15:15:30.422543 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:30.422378    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk" event={"ID":"13834160-216c-420e-ad87-5b9a1a780b03","Type":"ContainerStarted","Data":"4410fc513a5b5d51ae0ad1ad07e91c9c565207e896946af3dde1838dc98bff37"}
Jul 09 15:15:30.442326 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:30.442274    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk" podStartSLOduration=1.23255154 podStartE2EDuration="1.442258133s" totalImagesPullingTime="209.706593ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:15:30.440469221 +0000 UTC m=+780.773899191" watchObservedRunningTime="2026-07-09 15:15:30.442258133 +0000 UTC m=+780.775688077"
Jul 09 15:15:32.069492 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:32.069449    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:15:39.974925 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:39.974891    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:39.975303 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:39.974978    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:39.987078 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:39.987054    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:15:40.477786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:15:40.477758    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk"
Jul 09 15:16:00.012228 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:00.012146    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw"]
Jul 09 15:16:00.012660 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:00.012375    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw" podUID="f2a2887f-fc1b-4a8c-bfce-957dfc232998" containerName="cleanup" containerID="cri-o://21eb8b7381312493f0f5e7355e2dbc1111e55b52b97f025268380f9811b824aa" gracePeriod=30
Jul 09 15:16:30.201403 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:30.201378    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw"
Jul 09 15:16:30.287344 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:30.287263    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwvh6\" (UniqueName: \"kubernetes.io/projected/f2a2887f-fc1b-4a8c-bfce-957dfc232998-kube-api-access-rwvh6\") pod \"f2a2887f-fc1b-4a8c-bfce-957dfc232998\" (UID: \"f2a2887f-fc1b-4a8c-bfce-957dfc232998\") "
Jul 09 15:16:30.289371 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:30.289341    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2a2887f-fc1b-4a8c-bfce-957dfc232998-kube-api-access-rwvh6" (OuterVolumeSpecName: "kube-api-access-rwvh6") pod "f2a2887f-fc1b-4a8c-bfce-957dfc232998" (UID: "f2a2887f-fc1b-4a8c-bfce-957dfc232998"). InnerVolumeSpecName "kube-api-access-rwvh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:16:30.388242 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:30.388206    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwvh6\" (UniqueName: \"kubernetes.io/projected/f2a2887f-fc1b-4a8c-bfce-957dfc232998-kube-api-access-rwvh6\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:16:30.700875 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:30.700841    2579 generic.go:356] "Generic (PLEG): container finished" podID="f2a2887f-fc1b-4a8c-bfce-957dfc232998" containerID="21eb8b7381312493f0f5e7355e2dbc1111e55b52b97f025268380f9811b824aa" exitCode=137
Jul 09 15:16:30.701040 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:30.700884    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw"
Jul 09 15:16:30.701040 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:30.700903    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw" event={"ID":"f2a2887f-fc1b-4a8c-bfce-957dfc232998","Type":"ContainerDied","Data":"21eb8b7381312493f0f5e7355e2dbc1111e55b52b97f025268380f9811b824aa"}
Jul 09 15:16:30.701040 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:30.700948    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw" event={"ID":"f2a2887f-fc1b-4a8c-bfce-957dfc232998","Type":"ContainerDied","Data":"b9b121795334f87c6440bf3310dc7a561f3490d828ecd992d25c3238a778548f"}
Jul 09 15:16:30.701040 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:30.700969    2579 scope.go:117] "RemoveContainer" containerID="21eb8b7381312493f0f5e7355e2dbc1111e55b52b97f025268380f9811b824aa"
Jul 09 15:16:30.710185 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:30.710168    2579 scope.go:117] "RemoveContainer" containerID="21eb8b7381312493f0f5e7355e2dbc1111e55b52b97f025268380f9811b824aa"
Jul 09 15:16:30.710468 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:16:30.710448    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21eb8b7381312493f0f5e7355e2dbc1111e55b52b97f025268380f9811b824aa\": container with ID starting with 21eb8b7381312493f0f5e7355e2dbc1111e55b52b97f025268380f9811b824aa not found: ID does not exist" containerID="21eb8b7381312493f0f5e7355e2dbc1111e55b52b97f025268380f9811b824aa"
Jul 09 15:16:30.710544 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:30.710477    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21eb8b7381312493f0f5e7355e2dbc1111e55b52b97f025268380f9811b824aa"} err="failed to get container status \"21eb8b7381312493f0f5e7355e2dbc1111e55b52b97f025268380f9811b824aa\": rpc error: code = NotFound desc = could not find container \"21eb8b7381312493f0f5e7355e2dbc1111e55b52b97f025268380f9811b824aa\": container with ID starting with 21eb8b7381312493f0f5e7355e2dbc1111e55b52b97f025268380f9811b824aa not found: ID does not exist"
Jul 09 15:16:30.725994 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:30.725971    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw"]
Jul 09 15:16:30.731644 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:30.731626    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726835-gcxtw"]
Jul 09 15:16:32.231491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:16:32.231458    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2a2887f-fc1b-4a8c-bfce-957dfc232998" path="/var/lib/kubelet/pods/f2a2887f-fc1b-4a8c-bfce-957dfc232998/volumes"
Jul 09 15:17:04.563057 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:04.563024    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-79794498cf-snk7n"]
Jul 09 15:17:04.563551 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:04.563356    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2a2887f-fc1b-4a8c-bfce-957dfc232998" containerName="cleanup"
Jul 09 15:17:04.563551 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:04.563367    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2a2887f-fc1b-4a8c-bfce-957dfc232998" containerName="cleanup"
Jul 09 15:17:04.563551 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:04.563424    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2a2887f-fc1b-4a8c-bfce-957dfc232998" containerName="cleanup"
Jul 09 15:17:04.566362 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:04.566344    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79794498cf-snk7n"
Jul 09 15:17:04.575898 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:04.575867    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79794498cf-snk7n"]
Jul 09 15:17:04.688232 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:04.688188    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc-tls-cert\") pod \"authorino-79794498cf-snk7n\" (UID: \"f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc\") " pod="kuadrant-system/authorino-79794498cf-snk7n"
Jul 09 15:17:04.688421 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:04.688253    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9zlr\" (UniqueName: \"kubernetes.io/projected/f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc-kube-api-access-g9zlr\") pod \"authorino-79794498cf-snk7n\" (UID: \"f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc\") " pod="kuadrant-system/authorino-79794498cf-snk7n"
Jul 09 15:17:04.789385 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:04.789349    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9zlr\" (UniqueName: \"kubernetes.io/projected/f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc-kube-api-access-g9zlr\") pod \"authorino-79794498cf-snk7n\" (UID: \"f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc\") " pod="kuadrant-system/authorino-79794498cf-snk7n"
Jul 09 15:17:04.789578 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:04.789420    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc-tls-cert\") pod \"authorino-79794498cf-snk7n\" (UID: \"f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc\") " pod="kuadrant-system/authorino-79794498cf-snk7n"
Jul 09 15:17:04.791935 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:04.791905    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc-tls-cert\") pod \"authorino-79794498cf-snk7n\" (UID: \"f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc\") " pod="kuadrant-system/authorino-79794498cf-snk7n"
Jul 09 15:17:04.798093 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:04.798070    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9zlr\" (UniqueName: \"kubernetes.io/projected/f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc-kube-api-access-g9zlr\") pod \"authorino-79794498cf-snk7n\" (UID: \"f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc\") " pod="kuadrant-system/authorino-79794498cf-snk7n"
Jul 09 15:17:04.876034 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:04.875949    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-79794498cf-snk7n"
Jul 09 15:17:05.015665 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:05.015635    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-79794498cf-snk7n"]
Jul 09 15:17:05.017304 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:17:05.017274    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf77b8e53_16a8_4c87_b8c1_6ec750cbfdfc.slice/crio-40135dea9f65c106be1e3f9498a2d7add2bade7bdeea1f27cc175ee8f3c023b8 WatchSource:0}: Error finding container 40135dea9f65c106be1e3f9498a2d7add2bade7bdeea1f27cc175ee8f3c023b8: Status 404 returned error can't find the container with id 40135dea9f65c106be1e3f9498a2d7add2bade7bdeea1f27cc175ee8f3c023b8
Jul 09 15:17:05.865272 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:05.865159    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79794498cf-snk7n" event={"ID":"f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc","Type":"ContainerStarted","Data":"58ba87d27a17574d15b0578a0501c17aaf3beb2459cd421f7758ab693c8426c5"}
Jul 09 15:17:05.865272 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:05.865207    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-79794498cf-snk7n" event={"ID":"f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc","Type":"ContainerStarted","Data":"40135dea9f65c106be1e3f9498a2d7add2bade7bdeea1f27cc175ee8f3c023b8"}
Jul 09 15:17:05.892055 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:05.891997    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-79794498cf-snk7n" podStartSLOduration=1.403418095 podStartE2EDuration="1.89198094s" totalImagesPullingTime="488.562845ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:17:05.888563283 +0000 UTC m=+876.221993248" watchObservedRunningTime="2026-07-09 15:17:05.89198094 +0000 UTC m=+876.225410905"
Jul 09 15:17:05.927711 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:05.927649    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-676cdbf88b-njrrn"]
Jul 09 15:17:05.928421 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:05.928385    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-676cdbf88b-njrrn" podUID="8299963e-2655-4b33-b866-171695e01e19" containerName="authorino" containerID="cri-o://f7c7b0ebdd580e4a28caaa6f3be278bea23616389c24cdd67db1af06187b0051" gracePeriod=30
Jul 09 15:17:06.204252 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.204226    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-676cdbf88b-njrrn"
Jul 09 15:17:06.301974 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.301934    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6hrc\" (UniqueName: \"kubernetes.io/projected/8299963e-2655-4b33-b866-171695e01e19-kube-api-access-j6hrc\") pod \"8299963e-2655-4b33-b866-171695e01e19\" (UID: \"8299963e-2655-4b33-b866-171695e01e19\") "
Jul 09 15:17:06.301974 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.301980    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8299963e-2655-4b33-b866-171695e01e19-tls-cert\") pod \"8299963e-2655-4b33-b866-171695e01e19\" (UID: \"8299963e-2655-4b33-b866-171695e01e19\") "
Jul 09 15:17:06.304623 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.304576    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8299963e-2655-4b33-b866-171695e01e19-kube-api-access-j6hrc" (OuterVolumeSpecName: "kube-api-access-j6hrc") pod "8299963e-2655-4b33-b866-171695e01e19" (UID: "8299963e-2655-4b33-b866-171695e01e19"). InnerVolumeSpecName "kube-api-access-j6hrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:17:06.314540 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.314509    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8299963e-2655-4b33-b866-171695e01e19-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "8299963e-2655-4b33-b866-171695e01e19" (UID: "8299963e-2655-4b33-b866-171695e01e19"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:17:06.403399 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.403364    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8299963e-2655-4b33-b866-171695e01e19-tls-cert\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:17:06.403399 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.403392    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6hrc\" (UniqueName: \"kubernetes.io/projected/8299963e-2655-4b33-b866-171695e01e19-kube-api-access-j6hrc\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:17:06.871363 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.871325    2579 generic.go:356] "Generic (PLEG): container finished" podID="8299963e-2655-4b33-b866-171695e01e19" containerID="f7c7b0ebdd580e4a28caaa6f3be278bea23616389c24cdd67db1af06187b0051" exitCode=0
Jul 09 15:17:06.871894 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.871388    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-676cdbf88b-njrrn"
Jul 09 15:17:06.871894 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.871407    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-676cdbf88b-njrrn" event={"ID":"8299963e-2655-4b33-b866-171695e01e19","Type":"ContainerDied","Data":"f7c7b0ebdd580e4a28caaa6f3be278bea23616389c24cdd67db1af06187b0051"}
Jul 09 15:17:06.871894 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.871452    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-676cdbf88b-njrrn" event={"ID":"8299963e-2655-4b33-b866-171695e01e19","Type":"ContainerDied","Data":"efb2fae276832b4331fabb87e0735954d602bfd60d5f10cc05cddca286199e28"}
Jul 09 15:17:06.871894 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.871471    2579 scope.go:117] "RemoveContainer" containerID="f7c7b0ebdd580e4a28caaa6f3be278bea23616389c24cdd67db1af06187b0051"
Jul 09 15:17:06.881367 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.881343    2579 scope.go:117] "RemoveContainer" containerID="f7c7b0ebdd580e4a28caaa6f3be278bea23616389c24cdd67db1af06187b0051"
Jul 09 15:17:06.881737 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:17:06.881709    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7c7b0ebdd580e4a28caaa6f3be278bea23616389c24cdd67db1af06187b0051\": container with ID starting with f7c7b0ebdd580e4a28caaa6f3be278bea23616389c24cdd67db1af06187b0051 not found: ID does not exist" containerID="f7c7b0ebdd580e4a28caaa6f3be278bea23616389c24cdd67db1af06187b0051"
Jul 09 15:17:06.881865 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.881748    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7c7b0ebdd580e4a28caaa6f3be278bea23616389c24cdd67db1af06187b0051"} err="failed to get container status \"f7c7b0ebdd580e4a28caaa6f3be278bea23616389c24cdd67db1af06187b0051\": rpc error: code = NotFound desc = could not find container \"f7c7b0ebdd580e4a28caaa6f3be278bea23616389c24cdd67db1af06187b0051\": container with ID starting with f7c7b0ebdd580e4a28caaa6f3be278bea23616389c24cdd67db1af06187b0051 not found: ID does not exist"
Jul 09 15:17:06.895374 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.895340    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-676cdbf88b-njrrn"]
Jul 09 15:17:06.902204 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:06.902175    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-676cdbf88b-njrrn"]
Jul 09 15:17:08.231791 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:08.231751    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8299963e-2655-4b33-b866-171695e01e19" path="/var/lib/kubelet/pods/8299963e-2655-4b33-b866-171695e01e19/volumes"
Jul 09 15:17:30.116913 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:30.116828    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:17:30.119016 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:30.118994    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:17:35.151278 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:35.151242    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:17:46.554551 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:46.554516    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:17:56.356329 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:17:56.356294    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:18:05.348910 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:18:05.348871    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:18:16.454767 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:18:16.454730    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:18:44.275383 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:18:44.275345    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:18:57.395459 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:18:57.395375    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:19:38.687512 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:19:38.687476    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:19:48.313494 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:19:48.313450    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:19:58.575880 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:19:58.575844    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:20:06.677283 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:20:06.677247    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:20:18.769455 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:20:18.769414    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:20:30.064520 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:20:30.064484    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:20:41.972423 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:20:41.972391    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:20:52.265617 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:20:52.265577    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:21:13.561885 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:21:13.561852    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:21:24.062352 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:21:24.062314    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:21:49.759219 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:21:49.759171    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:22:06.456334 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:22:06.456243    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:22:18.252799 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:22:18.252762    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:22:28.268649 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:22:28.268609    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:22:30.152847 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:22:30.152816    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:22:30.154538 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:22:30.154518    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:22:38.060067 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:22:38.060030    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:22:47.362072 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:22:47.362028    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:22:55.755451 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:22:55.755407    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:23:21.561980 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:23:21.561941    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:24:54.352317 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:24:54.352235    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:25:02.267386 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:25:02.267347    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:25:10.867767 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:25:10.867720    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:25:22.879524 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:25:22.879483    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:25:35.058780 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:25:35.058740    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:25:44.462153 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:25:44.462112    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:25:53.766023 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:25:53.765985    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:26:04.761252 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:26:04.761215    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:26:15.682372 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:26:15.682331    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:26:24.774832 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:26:24.774741    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:26:35.971139 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:26:35.971099    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:26:57.463292 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:26:57.463246    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:27:07.462779 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:27:07.462741    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:27:17.799865 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:27:17.799823    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:27:27.366799 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:27:27.366760    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:27:30.182879 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:27:30.182842    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:27:30.185756 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:27:30.185732    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:27:34.154058 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:27:34.154009    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:27:50.666312 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:27:50.666278    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:28:00.966987 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:28:00.966914    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:28:10.889615 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:28:10.889567    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:28:21.473714 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:28:21.472879    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:28:31.274280 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:28:31.274244    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:28:41.168857 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:28:41.168817    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:28:52.783327 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:28:52.783283    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:29:09.960127 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:29:09.960088    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:29:19.780840 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:29:19.780806    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:29:26.571608 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:29:26.571525    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:29:38.082976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:29:38.082941    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:29:47.368371 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:29:47.368330    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:29:59.766360 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:29:59.766324    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:30:00.154461 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:00.154384    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp"]
Jul 09 15:30:00.154770 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:00.154755    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8299963e-2655-4b33-b866-171695e01e19" containerName="authorino"
Jul 09 15:30:00.154821 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:00.154772    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8299963e-2655-4b33-b866-171695e01e19" containerName="authorino"
Jul 09 15:30:00.154855 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:00.154827    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8299963e-2655-4b33-b866-171695e01e19" containerName="authorino"
Jul 09 15:30:00.157759 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:00.157740    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp"
Jul 09 15:30:00.218358 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:00.218326    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp"]
Jul 09 15:30:00.318412 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:00.318367    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr8kn\" (UniqueName: \"kubernetes.io/projected/e99604cb-ab23-47e1-a923-9a19efbbe549-kube-api-access-mr8kn\") pod \"maas-api-key-cleanup-29726850-fjrmp\" (UID: \"e99604cb-ab23-47e1-a923-9a19efbbe549\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp"
Jul 09 15:30:00.419582 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:00.419543    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr8kn\" (UniqueName: \"kubernetes.io/projected/e99604cb-ab23-47e1-a923-9a19efbbe549-kube-api-access-mr8kn\") pod \"maas-api-key-cleanup-29726850-fjrmp\" (UID: \"e99604cb-ab23-47e1-a923-9a19efbbe549\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp"
Jul 09 15:30:00.432766 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:00.432735    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr8kn\" (UniqueName: \"kubernetes.io/projected/e99604cb-ab23-47e1-a923-9a19efbbe549-kube-api-access-mr8kn\") pod \"maas-api-key-cleanup-29726850-fjrmp\" (UID: \"e99604cb-ab23-47e1-a923-9a19efbbe549\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp"
Jul 09 15:30:00.467296 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:00.467262    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp"
Jul 09 15:30:00.613422 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:00.613397    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp"]
Jul 09 15:30:00.615550 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:30:00.615515    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode99604cb_ab23_47e1_a923_9a19efbbe549.slice/crio-93541c27f0c2031739120ac724de208359eb413fae34abb7de0ced66d74a8047 WatchSource:0}: Error finding container 93541c27f0c2031739120ac724de208359eb413fae34abb7de0ced66d74a8047: Status 404 returned error can't find the container with id 93541c27f0c2031739120ac724de208359eb413fae34abb7de0ced66d74a8047
Jul 09 15:30:01.512024 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:01.511990    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp" event={"ID":"e99604cb-ab23-47e1-a923-9a19efbbe549","Type":"ContainerStarted","Data":"7d5196071024ad0d76407145ccb91c8412a8875760d0f4ebdfd4a412e4ed4107"}
Jul 09 15:30:01.512024 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:01.512041    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp" event={"ID":"e99604cb-ab23-47e1-a923-9a19efbbe549","Type":"ContainerStarted","Data":"93541c27f0c2031739120ac724de208359eb413fae34abb7de0ced66d74a8047"}
Jul 09 15:30:01.533638 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:01.533578    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp" podStartSLOduration=1.533557032 podStartE2EDuration="1.533557032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:30:01.533531393 +0000 UTC m=+1651.866961356" watchObservedRunningTime="2026-07-09 15:30:01.533557032 +0000 UTC m=+1651.866986996"
Jul 09 15:30:12.060148 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:12.060100    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:30:26.390460 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:26.390427    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:30:41.851500 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:41.851461    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"]
Jul 09 15:30:41.854823 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:41.854794    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"
Jul 09 15:30:41.867474 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:30:41.867444    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-e2e-ait-1829e532-serving-cert\" is forbidden: User \"system:node:ip-10-0-129-152.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-129-152.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1829e532-serving-cert\"" type="*v1.Secret"
Jul 09 15:30:41.912863 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:41.912820    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"]
Jul 09 15:30:41.976870 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:41.976836    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzdqb\" (UniqueName: \"kubernetes.io/projected/a71f5ef7-971f-4c17-8e7e-0b6627a829be-kube-api-access-rzdqb\") pod \"maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl\" (UID: \"a71f5ef7-971f-4c17-8e7e-0b6627a829be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"
Jul 09 15:30:41.976870 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:41.976873    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a71f5ef7-971f-4c17-8e7e-0b6627a829be-maas-api-tls\") pod \"maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl\" (UID: \"a71f5ef7-971f-4c17-8e7e-0b6627a829be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"
Jul 09 15:30:41.977094 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:41.976972    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a71f5ef7-971f-4c17-8e7e-0b6627a829be-openshift-service-ca\") pod \"maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl\" (UID: \"a71f5ef7-971f-4c17-8e7e-0b6627a829be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"
Jul 09 15:30:42.077604 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:42.077566    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzdqb\" (UniqueName: \"kubernetes.io/projected/a71f5ef7-971f-4c17-8e7e-0b6627a829be-kube-api-access-rzdqb\") pod \"maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl\" (UID: \"a71f5ef7-971f-4c17-8e7e-0b6627a829be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"
Jul 09 15:30:42.077604 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:42.077607    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a71f5ef7-971f-4c17-8e7e-0b6627a829be-maas-api-tls\") pod \"maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl\" (UID: \"a71f5ef7-971f-4c17-8e7e-0b6627a829be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"
Jul 09 15:30:42.077900 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:42.077660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a71f5ef7-971f-4c17-8e7e-0b6627a829be-openshift-service-ca\") pod \"maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl\" (UID: \"a71f5ef7-971f-4c17-8e7e-0b6627a829be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"
Jul 09 15:30:42.078273 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:42.078253    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a71f5ef7-971f-4c17-8e7e-0b6627a829be-openshift-service-ca\") pod \"maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl\" (UID: \"a71f5ef7-971f-4c17-8e7e-0b6627a829be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"
Jul 09 15:30:42.099856 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:42.099828    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzdqb\" (UniqueName: \"kubernetes.io/projected/a71f5ef7-971f-4c17-8e7e-0b6627a829be-kube-api-access-rzdqb\") pod \"maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl\" (UID: \"a71f5ef7-971f-4c17-8e7e-0b6627a829be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"
Jul 09 15:30:42.978955 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:42.978922    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1829e532-serving-cert\""
Jul 09 15:30:42.990868 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:42.990833    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a71f5ef7-971f-4c17-8e7e-0b6627a829be-maas-api-tls\") pod \"maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl\" (UID: \"a71f5ef7-971f-4c17-8e7e-0b6627a829be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"
Jul 09 15:30:43.065853 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:43.065813    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"
Jul 09 15:30:43.210647 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:43.210619    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"]
Jul 09 15:30:43.211464 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:30:43.211426    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda71f5ef7_971f_4c17_8e7e_0b6627a829be.slice/crio-577f82a96e3dfe5d976d634851038f2439690ab138e79157f142ca041d72756b WatchSource:0}: Error finding container 577f82a96e3dfe5d976d634851038f2439690ab138e79157f142ca041d72756b: Status 404 returned error can't find the container with id 577f82a96e3dfe5d976d634851038f2439690ab138e79157f142ca041d72756b
Jul 09 15:30:43.212854 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:43.212835    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 15:30:43.709554 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:43.709519    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl" event={"ID":"a71f5ef7-971f-4c17-8e7e-0b6627a829be","Type":"ContainerStarted","Data":"577f82a96e3dfe5d976d634851038f2439690ab138e79157f142ca041d72756b"}
Jul 09 15:30:44.715542 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:44.715500    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl" event={"ID":"a71f5ef7-971f-4c17-8e7e-0b6627a829be","Type":"ContainerStarted","Data":"9f0d7738e5bc49329b7d7886a64f33c980a649990af8c7777dade79f6e83c14f"}
Jul 09 15:30:44.715944 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:44.715572    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"
Jul 09 15:30:44.767471 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:44.767431    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:30:47.768244 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:47.768187    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl" podStartSLOduration=6.120904981 podStartE2EDuration="6.768170726s" totalImagesPullingTime="647.265745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:30:44.801941218 +0000 UTC m=+1695.135371192" watchObservedRunningTime="2026-07-09 15:30:47.768170726 +0000 UTC m=+1698.101600711"
Jul 09 15:30:47.769367 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:47.769344    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"]
Jul 09 15:30:47.769574 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:47.769555    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl" podUID="a71f5ef7-971f-4c17-8e7e-0b6627a829be" containerName="maas-api" containerID="cri-o://9f0d7738e5bc49329b7d7886a64f33c980a649990af8c7777dade79f6e83c14f" gracePeriod=30
Jul 09 15:30:48.014327 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.014304    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"
Jul 09 15:30:48.133051 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.132962    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzdqb\" (UniqueName: \"kubernetes.io/projected/a71f5ef7-971f-4c17-8e7e-0b6627a829be-kube-api-access-rzdqb\") pod \"a71f5ef7-971f-4c17-8e7e-0b6627a829be\" (UID: \"a71f5ef7-971f-4c17-8e7e-0b6627a829be\") "
Jul 09 15:30:48.133051 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.133012    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a71f5ef7-971f-4c17-8e7e-0b6627a829be-maas-api-tls\") pod \"a71f5ef7-971f-4c17-8e7e-0b6627a829be\" (UID: \"a71f5ef7-971f-4c17-8e7e-0b6627a829be\") "
Jul 09 15:30:48.133255 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.133078    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a71f5ef7-971f-4c17-8e7e-0b6627a829be-openshift-service-ca\") pod \"a71f5ef7-971f-4c17-8e7e-0b6627a829be\" (UID: \"a71f5ef7-971f-4c17-8e7e-0b6627a829be\") "
Jul 09 15:30:48.133497 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.133470    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a71f5ef7-971f-4c17-8e7e-0b6627a829be-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a71f5ef7-971f-4c17-8e7e-0b6627a829be" (UID: "a71f5ef7-971f-4c17-8e7e-0b6627a829be"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:30:48.135062 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.135039    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a71f5ef7-971f-4c17-8e7e-0b6627a829be-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a71f5ef7-971f-4c17-8e7e-0b6627a829be" (UID: "a71f5ef7-971f-4c17-8e7e-0b6627a829be"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:30:48.135125 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.135091    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a71f5ef7-971f-4c17-8e7e-0b6627a829be-kube-api-access-rzdqb" (OuterVolumeSpecName: "kube-api-access-rzdqb") pod "a71f5ef7-971f-4c17-8e7e-0b6627a829be" (UID: "a71f5ef7-971f-4c17-8e7e-0b6627a829be"). InnerVolumeSpecName "kube-api-access-rzdqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:30:48.234023 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.233989    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzdqb\" (UniqueName: \"kubernetes.io/projected/a71f5ef7-971f-4c17-8e7e-0b6627a829be-kube-api-access-rzdqb\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:30:48.234023 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.234021    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a71f5ef7-971f-4c17-8e7e-0b6627a829be-maas-api-tls\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:30:48.234214 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.234036    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a71f5ef7-971f-4c17-8e7e-0b6627a829be-openshift-service-ca\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:30:48.737417 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.737384    2579 generic.go:356] "Generic (PLEG): container finished" podID="a71f5ef7-971f-4c17-8e7e-0b6627a829be" containerID="9f0d7738e5bc49329b7d7886a64f33c980a649990af8c7777dade79f6e83c14f" exitCode=0
Jul 09 15:30:48.737594 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.737428    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"
Jul 09 15:30:48.737594 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.737430    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl" event={"ID":"a71f5ef7-971f-4c17-8e7e-0b6627a829be","Type":"ContainerDied","Data":"9f0d7738e5bc49329b7d7886a64f33c980a649990af8c7777dade79f6e83c14f"}
Jul 09 15:30:48.737594 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.737566    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl" event={"ID":"a71f5ef7-971f-4c17-8e7e-0b6627a829be","Type":"ContainerDied","Data":"577f82a96e3dfe5d976d634851038f2439690ab138e79157f142ca041d72756b"}
Jul 09 15:30:48.737594 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.737594    2579 scope.go:117] "RemoveContainer" containerID="9f0d7738e5bc49329b7d7886a64f33c980a649990af8c7777dade79f6e83c14f"
Jul 09 15:30:48.745964 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.745946    2579 scope.go:117] "RemoveContainer" containerID="9f0d7738e5bc49329b7d7886a64f33c980a649990af8c7777dade79f6e83c14f"
Jul 09 15:30:48.746233 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:30:48.746217    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f0d7738e5bc49329b7d7886a64f33c980a649990af8c7777dade79f6e83c14f\": container with ID starting with 9f0d7738e5bc49329b7d7886a64f33c980a649990af8c7777dade79f6e83c14f not found: ID does not exist" containerID="9f0d7738e5bc49329b7d7886a64f33c980a649990af8c7777dade79f6e83c14f"
Jul 09 15:30:48.746274 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.746241    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f0d7738e5bc49329b7d7886a64f33c980a649990af8c7777dade79f6e83c14f"} err="failed to get container status \"9f0d7738e5bc49329b7d7886a64f33c980a649990af8c7777dade79f6e83c14f\": rpc error: code = NotFound desc = could not find container \"9f0d7738e5bc49329b7d7886a64f33c980a649990af8c7777dade79f6e83c14f\": container with ID starting with 9f0d7738e5bc49329b7d7886a64f33c980a649990af8c7777dade79f6e83c14f not found: ID does not exist"
Jul 09 15:30:48.757117 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.757092    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"]
Jul 09 15:30:48.760531 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:48.760509    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1829e532-664f6ccb6f-wpccl"]
Jul 09 15:30:50.230927 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:50.230889    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a71f5ef7-971f-4c17-8e7e-0b6627a829be" path="/var/lib/kubelet/pods/a71f5ef7-971f-4c17-8e7e-0b6627a829be/volumes"
Jul 09 15:30:55.150263 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.150225    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"]
Jul 09 15:30:55.150737 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.150717    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a71f5ef7-971f-4c17-8e7e-0b6627a829be" containerName="maas-api"
Jul 09 15:30:55.150819 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.150739    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a71f5ef7-971f-4c17-8e7e-0b6627a829be" containerName="maas-api"
Jul 09 15:30:55.150876 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.150861    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a71f5ef7-971f-4c17-8e7e-0b6627a829be" containerName="maas-api"
Jul 09 15:30:55.154958 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.154936    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"
Jul 09 15:30:55.158168 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.158148    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-40e15ce5-serving-cert\""
Jul 09 15:30:55.184835 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.184806    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"]
Jul 09 15:30:55.297099 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.297053    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjncr\" (UniqueName: \"kubernetes.io/projected/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-kube-api-access-sjncr\") pod \"maas-api-e2e-ait-40e15ce5-645966b996-76j44\" (UID: \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"
Jul 09 15:30:55.297099 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.297106    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-openshift-service-ca\") pod \"maas-api-e2e-ait-40e15ce5-645966b996-76j44\" (UID: \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"
Jul 09 15:30:55.297321 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.297201    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-maas-api-tls\") pod \"maas-api-e2e-ait-40e15ce5-645966b996-76j44\" (UID: \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"
Jul 09 15:30:55.398489 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.398446    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-maas-api-tls\") pod \"maas-api-e2e-ait-40e15ce5-645966b996-76j44\" (UID: \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"
Jul 09 15:30:55.398662 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:30:55.398600    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-serving-cert: secret "maas-api-e2e-ait-40e15ce5-serving-cert" not found
Jul 09 15:30:55.398662 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.398610    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjncr\" (UniqueName: \"kubernetes.io/projected/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-kube-api-access-sjncr\") pod \"maas-api-e2e-ait-40e15ce5-645966b996-76j44\" (UID: \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"
Jul 09 15:30:55.398662 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.398646    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-openshift-service-ca\") pod \"maas-api-e2e-ait-40e15ce5-645966b996-76j44\" (UID: \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"
Jul 09 15:30:55.398807 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:30:55.398672    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-maas-api-tls podName:1a8c29c2-7f05-430e-bfec-4d4cdcb401c8 nodeName:}" failed. No retries permitted until 2026-07-09 15:30:55.898652575 +0000 UTC m=+1706.232082517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-maas-api-tls") pod "maas-api-e2e-ait-40e15ce5-645966b996-76j44" (UID: "1a8c29c2-7f05-430e-bfec-4d4cdcb401c8") : secret "maas-api-e2e-ait-40e15ce5-serving-cert" not found
Jul 09 15:30:55.399250 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.399232    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-openshift-service-ca\") pod \"maas-api-e2e-ait-40e15ce5-645966b996-76j44\" (UID: \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"
Jul 09 15:30:55.459558 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.459521    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjncr\" (UniqueName: \"kubernetes.io/projected/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-kube-api-access-sjncr\") pod \"maas-api-e2e-ait-40e15ce5-645966b996-76j44\" (UID: \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"
Jul 09 15:30:55.904077 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.903987    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-maas-api-tls\") pod \"maas-api-e2e-ait-40e15ce5-645966b996-76j44\" (UID: \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"
Jul 09 15:30:55.906358 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:55.906331    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-maas-api-tls\") pod \"maas-api-e2e-ait-40e15ce5-645966b996-76j44\" (UID: \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"
Jul 09 15:30:56.064906 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:56.064874    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"
Jul 09 15:30:56.212582 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:56.212554    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"]
Jul 09 15:30:56.214436 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:30:56.214407    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a8c29c2_7f05_430e_bfec_4d4cdcb401c8.slice/crio-7fc291e77b0c4dba3441f1a65ebc1da6503b205f5c770c3770f1630cc20dfefb WatchSource:0}: Error finding container 7fc291e77b0c4dba3441f1a65ebc1da6503b205f5c770c3770f1630cc20dfefb: Status 404 returned error can't find the container with id 7fc291e77b0c4dba3441f1a65ebc1da6503b205f5c770c3770f1630cc20dfefb
Jul 09 15:30:56.782468 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:56.782437    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44" event={"ID":"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8","Type":"ContainerStarted","Data":"06356b7cc3983bd61598cbf9d3376f4a5608d8ad3933aec2ba185917f0e348fe"}
Jul 09 15:30:56.782623 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:56.782486    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44" event={"ID":"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8","Type":"ContainerStarted","Data":"7fc291e77b0c4dba3441f1a65ebc1da6503b205f5c770c3770f1630cc20dfefb"}
Jul 09 15:30:56.782623 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:56.782539    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"
Jul 09 15:30:56.816374 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:56.816323    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44" podStartSLOduration=1.410421211 podStartE2EDuration="1.816307366s" totalImagesPullingTime="405.886155ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:30:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:30:56.8148806 +0000 UTC m=+1707.148310566" watchObservedRunningTime="2026-07-09 15:30:56.816307366 +0000 UTC m=+1707.149737329"
Jul 09 15:30:57.890082 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:30:57.890046    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:31:00.011013 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:00.010981    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp"]
Jul 09 15:31:00.011388 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:00.011158    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp" podUID="e99604cb-ab23-47e1-a923-9a19efbbe549" containerName="cleanup" containerID="cri-o://7d5196071024ad0d76407145ccb91c8412a8875760d0f4ebdfd4a412e4ed4107" gracePeriod=30
Jul 09 15:31:00.771136 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:00.771103    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"]
Jul 09 15:31:00.771344 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:00.771306    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44" podUID="1a8c29c2-7f05-430e-bfec-4d4cdcb401c8" containerName="maas-api" containerID="cri-o://06356b7cc3983bd61598cbf9d3376f4a5608d8ad3933aec2ba185917f0e348fe" gracePeriod=30
Jul 09 15:31:01.016723 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.016697    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"
Jul 09 15:31:01.151791 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.151675    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-openshift-service-ca\") pod \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\" (UID: \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\") "
Jul 09 15:31:01.151957 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.151802    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjncr\" (UniqueName: \"kubernetes.io/projected/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-kube-api-access-sjncr\") pod \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\" (UID: \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\") "
Jul 09 15:31:01.151957 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.151865    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-maas-api-tls\") pod \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\" (UID: \"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8\") "
Jul 09 15:31:01.152094 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.152066    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1a8c29c2-7f05-430e-bfec-4d4cdcb401c8" (UID: "1a8c29c2-7f05-430e-bfec-4d4cdcb401c8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:31:01.153877 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.153847    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-kube-api-access-sjncr" (OuterVolumeSpecName: "kube-api-access-sjncr") pod "1a8c29c2-7f05-430e-bfec-4d4cdcb401c8" (UID: "1a8c29c2-7f05-430e-bfec-4d4cdcb401c8"). InnerVolumeSpecName "kube-api-access-sjncr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:31:01.153982 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.153871    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1a8c29c2-7f05-430e-bfec-4d4cdcb401c8" (UID: "1a8c29c2-7f05-430e-bfec-4d4cdcb401c8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:31:01.253134 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.253088    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjncr\" (UniqueName: \"kubernetes.io/projected/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-kube-api-access-sjncr\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:31:01.253134 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.253125    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-maas-api-tls\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:31:01.253134 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.253137    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8-openshift-service-ca\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:31:01.806317 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.806282    2579 generic.go:356] "Generic (PLEG): container finished" podID="1a8c29c2-7f05-430e-bfec-4d4cdcb401c8" containerID="06356b7cc3983bd61598cbf9d3376f4a5608d8ad3933aec2ba185917f0e348fe" exitCode=0
Jul 09 15:31:01.806519 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.806328    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"
Jul 09 15:31:01.806519 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.806368    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44" event={"ID":"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8","Type":"ContainerDied","Data":"06356b7cc3983bd61598cbf9d3376f4a5608d8ad3933aec2ba185917f0e348fe"}
Jul 09 15:31:01.806519 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.806426    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44" event={"ID":"1a8c29c2-7f05-430e-bfec-4d4cdcb401c8","Type":"ContainerDied","Data":"7fc291e77b0c4dba3441f1a65ebc1da6503b205f5c770c3770f1630cc20dfefb"}
Jul 09 15:31:01.806519 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.806456    2579 scope.go:117] "RemoveContainer" containerID="06356b7cc3983bd61598cbf9d3376f4a5608d8ad3933aec2ba185917f0e348fe"
Jul 09 15:31:01.815031 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.815014    2579 scope.go:117] "RemoveContainer" containerID="06356b7cc3983bd61598cbf9d3376f4a5608d8ad3933aec2ba185917f0e348fe"
Jul 09 15:31:01.815267 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:31:01.815249    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06356b7cc3983bd61598cbf9d3376f4a5608d8ad3933aec2ba185917f0e348fe\": container with ID starting with 06356b7cc3983bd61598cbf9d3376f4a5608d8ad3933aec2ba185917f0e348fe not found: ID does not exist" containerID="06356b7cc3983bd61598cbf9d3376f4a5608d8ad3933aec2ba185917f0e348fe"
Jul 09 15:31:01.815308 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.815273    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06356b7cc3983bd61598cbf9d3376f4a5608d8ad3933aec2ba185917f0e348fe"} err="failed to get container status \"06356b7cc3983bd61598cbf9d3376f4a5608d8ad3933aec2ba185917f0e348fe\": rpc error: code = NotFound desc = could not find container \"06356b7cc3983bd61598cbf9d3376f4a5608d8ad3933aec2ba185917f0e348fe\": container with ID starting with 06356b7cc3983bd61598cbf9d3376f4a5608d8ad3933aec2ba185917f0e348fe not found: ID does not exist"
Jul 09 15:31:01.830203 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.830179    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"]
Jul 09 15:31:01.836228 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:01.836207    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-40e15ce5-645966b996-76j44"]
Jul 09 15:31:02.230749 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:02.230708    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a8c29c2-7f05-430e-bfec-4d4cdcb401c8" path="/var/lib/kubelet/pods/1a8c29c2-7f05-430e-bfec-4d4cdcb401c8/volumes"
Jul 09 15:31:10.267120 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:10.267082    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:31:15.001667 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.001636    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"]
Jul 09 15:31:15.002074 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.001986    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a8c29c2-7f05-430e-bfec-4d4cdcb401c8" containerName="maas-api"
Jul 09 15:31:15.002074 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.001999    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a8c29c2-7f05-430e-bfec-4d4cdcb401c8" containerName="maas-api"
Jul 09 15:31:15.002074 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.002050    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a8c29c2-7f05-430e-bfec-4d4cdcb401c8" containerName="maas-api"
Jul 09 15:31:15.005160 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.005138    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"
Jul 09 15:31:15.009358 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.009339    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-14931b17-serving-cert\""
Jul 09 15:31:15.025176 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.025141    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"]
Jul 09 15:31:15.061376 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.061343    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3a33e97-4296-430c-813f-79e8249fd27c-maas-api-tls\") pod \"maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f\" (UID: \"a3a33e97-4296-430c-813f-79e8249fd27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"
Jul 09 15:31:15.061549 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.061391    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z6zm\" (UniqueName: \"kubernetes.io/projected/a3a33e97-4296-430c-813f-79e8249fd27c-kube-api-access-4z6zm\") pod \"maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f\" (UID: \"a3a33e97-4296-430c-813f-79e8249fd27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"
Jul 09 15:31:15.061549 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.061484    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3a33e97-4296-430c-813f-79e8249fd27c-openshift-service-ca\") pod \"maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f\" (UID: \"a3a33e97-4296-430c-813f-79e8249fd27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"
Jul 09 15:31:15.162187 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.162152    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3a33e97-4296-430c-813f-79e8249fd27c-maas-api-tls\") pod \"maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f\" (UID: \"a3a33e97-4296-430c-813f-79e8249fd27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"
Jul 09 15:31:15.162354 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.162198    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z6zm\" (UniqueName: \"kubernetes.io/projected/a3a33e97-4296-430c-813f-79e8249fd27c-kube-api-access-4z6zm\") pod \"maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f\" (UID: \"a3a33e97-4296-430c-813f-79e8249fd27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"
Jul 09 15:31:15.162354 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.162238    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3a33e97-4296-430c-813f-79e8249fd27c-openshift-service-ca\") pod \"maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f\" (UID: \"a3a33e97-4296-430c-813f-79e8249fd27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"
Jul 09 15:31:15.162804 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.162784    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3a33e97-4296-430c-813f-79e8249fd27c-openshift-service-ca\") pod \"maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f\" (UID: \"a3a33e97-4296-430c-813f-79e8249fd27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"
Jul 09 15:31:15.164547 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.164526    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3a33e97-4296-430c-813f-79e8249fd27c-maas-api-tls\") pod \"maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f\" (UID: \"a3a33e97-4296-430c-813f-79e8249fd27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"
Jul 09 15:31:15.180060 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.180027    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z6zm\" (UniqueName: \"kubernetes.io/projected/a3a33e97-4296-430c-813f-79e8249fd27c-kube-api-access-4z6zm\") pod \"maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f\" (UID: \"a3a33e97-4296-430c-813f-79e8249fd27c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"
Jul 09 15:31:15.315406 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.315327    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"
Jul 09 15:31:15.457385 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.457363    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"]
Jul 09 15:31:15.459546 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:31:15.459519    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a33e97_4296_430c_813f_79e8249fd27c.slice/crio-9434b71b0c9b079abb7230bb3bfc957170196a93c72e88e33acb5f233b80b50b WatchSource:0}: Error finding container 9434b71b0c9b079abb7230bb3bfc957170196a93c72e88e33acb5f233b80b50b: Status 404 returned error can't find the container with id 9434b71b0c9b079abb7230bb3bfc957170196a93c72e88e33acb5f233b80b50b
Jul 09 15:31:15.877385 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:15.877356    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f" event={"ID":"a3a33e97-4296-430c-813f-79e8249fd27c","Type":"ContainerStarted","Data":"9434b71b0c9b079abb7230bb3bfc957170196a93c72e88e33acb5f233b80b50b"}
Jul 09 15:31:16.884013 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:16.883976    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f" event={"ID":"a3a33e97-4296-430c-813f-79e8249fd27c","Type":"ContainerStarted","Data":"f7c17e39a17955596543cd92115c5bbd778e7edbaf503854a80f4a9f7cd1eac3"}
Jul 09 15:31:16.884406 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:16.884164    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"
Jul 09 15:31:16.910657 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:16.910609    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f" podStartSLOduration=2.519135008 podStartE2EDuration="2.910593972s" totalImagesPullingTime="391.458964ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:31:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:31:16.908405413 +0000 UTC m=+1727.241835378" watchObservedRunningTime="2026-07-09 15:31:16.910593972 +0000 UTC m=+1727.244023935"
Jul 09 15:31:18.326398 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:18.326360    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:31:19.992637 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:19.992607    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"]
Jul 09 15:31:19.993079 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:19.992820    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f" podUID="a3a33e97-4296-430c-813f-79e8249fd27c" containerName="maas-api" containerID="cri-o://f7c17e39a17955596543cd92115c5bbd778e7edbaf503854a80f4a9f7cd1eac3" gracePeriod=30
Jul 09 15:31:20.239061 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.239038    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"
Jul 09 15:31:20.308621 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.308550    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3a33e97-4296-430c-813f-79e8249fd27c-openshift-service-ca\") pod \"a3a33e97-4296-430c-813f-79e8249fd27c\" (UID: \"a3a33e97-4296-430c-813f-79e8249fd27c\") "
Jul 09 15:31:20.308621 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.308595    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3a33e97-4296-430c-813f-79e8249fd27c-maas-api-tls\") pod \"a3a33e97-4296-430c-813f-79e8249fd27c\" (UID: \"a3a33e97-4296-430c-813f-79e8249fd27c\") "
Jul 09 15:31:20.308835 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.308626    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4z6zm\" (UniqueName: \"kubernetes.io/projected/a3a33e97-4296-430c-813f-79e8249fd27c-kube-api-access-4z6zm\") pod \"a3a33e97-4296-430c-813f-79e8249fd27c\" (UID: \"a3a33e97-4296-430c-813f-79e8249fd27c\") "
Jul 09 15:31:20.308934 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.308905    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3a33e97-4296-430c-813f-79e8249fd27c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a3a33e97-4296-430c-813f-79e8249fd27c" (UID: "a3a33e97-4296-430c-813f-79e8249fd27c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:31:20.310653 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.310623    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3a33e97-4296-430c-813f-79e8249fd27c-kube-api-access-4z6zm" (OuterVolumeSpecName: "kube-api-access-4z6zm") pod "a3a33e97-4296-430c-813f-79e8249fd27c" (UID: "a3a33e97-4296-430c-813f-79e8249fd27c"). InnerVolumeSpecName "kube-api-access-4z6zm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:31:20.310780 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.310715    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3a33e97-4296-430c-813f-79e8249fd27c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a3a33e97-4296-430c-813f-79e8249fd27c" (UID: "a3a33e97-4296-430c-813f-79e8249fd27c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:31:20.409870 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.409831    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3a33e97-4296-430c-813f-79e8249fd27c-maas-api-tls\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:31:20.409870 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.409856    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z6zm\" (UniqueName: \"kubernetes.io/projected/a3a33e97-4296-430c-813f-79e8249fd27c-kube-api-access-4z6zm\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:31:20.409870 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.409867    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3a33e97-4296-430c-813f-79e8249fd27c-openshift-service-ca\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:31:20.903472 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.903436    2579 generic.go:356] "Generic (PLEG): container finished" podID="a3a33e97-4296-430c-813f-79e8249fd27c" containerID="f7c17e39a17955596543cd92115c5bbd778e7edbaf503854a80f4a9f7cd1eac3" exitCode=0
Jul 09 15:31:20.903644 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.903482    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"
Jul 09 15:31:20.903644 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.903511    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f" event={"ID":"a3a33e97-4296-430c-813f-79e8249fd27c","Type":"ContainerDied","Data":"f7c17e39a17955596543cd92115c5bbd778e7edbaf503854a80f4a9f7cd1eac3"}
Jul 09 15:31:20.903644 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.903580    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f" event={"ID":"a3a33e97-4296-430c-813f-79e8249fd27c","Type":"ContainerDied","Data":"9434b71b0c9b079abb7230bb3bfc957170196a93c72e88e33acb5f233b80b50b"}
Jul 09 15:31:20.903644 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.903602    2579 scope.go:117] "RemoveContainer" containerID="f7c17e39a17955596543cd92115c5bbd778e7edbaf503854a80f4a9f7cd1eac3"
Jul 09 15:31:20.921863 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.921842    2579 scope.go:117] "RemoveContainer" containerID="f7c17e39a17955596543cd92115c5bbd778e7edbaf503854a80f4a9f7cd1eac3"
Jul 09 15:31:20.922117 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:31:20.922098    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7c17e39a17955596543cd92115c5bbd778e7edbaf503854a80f4a9f7cd1eac3\": container with ID starting with f7c17e39a17955596543cd92115c5bbd778e7edbaf503854a80f4a9f7cd1eac3 not found: ID does not exist" containerID="f7c17e39a17955596543cd92115c5bbd778e7edbaf503854a80f4a9f7cd1eac3"
Jul 09 15:31:20.922158 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.922125    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7c17e39a17955596543cd92115c5bbd778e7edbaf503854a80f4a9f7cd1eac3"} err="failed to get container status \"f7c17e39a17955596543cd92115c5bbd778e7edbaf503854a80f4a9f7cd1eac3\": rpc error: code = NotFound desc = could not find container \"f7c17e39a17955596543cd92115c5bbd778e7edbaf503854a80f4a9f7cd1eac3\": container with ID starting with f7c17e39a17955596543cd92115c5bbd778e7edbaf503854a80f4a9f7cd1eac3 not found: ID does not exist"
Jul 09 15:31:20.945349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.945321    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"]
Jul 09 15:31:20.953926 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:20.953901    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-14931b17-7f4cf66985-n9h2f"]
Jul 09 15:31:22.230977 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:22.230943    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3a33e97-4296-430c-813f-79e8249fd27c" path="/var/lib/kubelet/pods/a3a33e97-4296-430c-813f-79e8249fd27c/volumes"
Jul 09 15:31:30.203352 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:30.203328    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp"
Jul 09 15:31:30.298304 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:30.298278    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mr8kn\" (UniqueName: \"kubernetes.io/projected/e99604cb-ab23-47e1-a923-9a19efbbe549-kube-api-access-mr8kn\") pod \"e99604cb-ab23-47e1-a923-9a19efbbe549\" (UID: \"e99604cb-ab23-47e1-a923-9a19efbbe549\") "
Jul 09 15:31:30.300323 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:30.300294    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e99604cb-ab23-47e1-a923-9a19efbbe549-kube-api-access-mr8kn" (OuterVolumeSpecName: "kube-api-access-mr8kn") pod "e99604cb-ab23-47e1-a923-9a19efbbe549" (UID: "e99604cb-ab23-47e1-a923-9a19efbbe549"). InnerVolumeSpecName "kube-api-access-mr8kn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:31:30.399747 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:30.399705    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mr8kn\" (UniqueName: \"kubernetes.io/projected/e99604cb-ab23-47e1-a923-9a19efbbe549-kube-api-access-mr8kn\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:31:30.954222 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:30.954184    2579 generic.go:356] "Generic (PLEG): container finished" podID="e99604cb-ab23-47e1-a923-9a19efbbe549" containerID="7d5196071024ad0d76407145ccb91c8412a8875760d0f4ebdfd4a412e4ed4107" exitCode=137
Jul 09 15:31:30.954407 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:30.954243    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp" event={"ID":"e99604cb-ab23-47e1-a923-9a19efbbe549","Type":"ContainerDied","Data":"7d5196071024ad0d76407145ccb91c8412a8875760d0f4ebdfd4a412e4ed4107"}
Jul 09 15:31:30.954407 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:30.954292    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp" event={"ID":"e99604cb-ab23-47e1-a923-9a19efbbe549","Type":"ContainerDied","Data":"93541c27f0c2031739120ac724de208359eb413fae34abb7de0ced66d74a8047"}
Jul 09 15:31:30.954407 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:30.954315    2579 scope.go:117] "RemoveContainer" containerID="7d5196071024ad0d76407145ccb91c8412a8875760d0f4ebdfd4a412e4ed4107"
Jul 09 15:31:30.954407 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:30.954245    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp"
Jul 09 15:31:30.963046 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:30.963028    2579 scope.go:117] "RemoveContainer" containerID="7d5196071024ad0d76407145ccb91c8412a8875760d0f4ebdfd4a412e4ed4107"
Jul 09 15:31:30.963360 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:31:30.963340    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d5196071024ad0d76407145ccb91c8412a8875760d0f4ebdfd4a412e4ed4107\": container with ID starting with 7d5196071024ad0d76407145ccb91c8412a8875760d0f4ebdfd4a412e4ed4107 not found: ID does not exist" containerID="7d5196071024ad0d76407145ccb91c8412a8875760d0f4ebdfd4a412e4ed4107"
Jul 09 15:31:30.963420 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:30.963376    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d5196071024ad0d76407145ccb91c8412a8875760d0f4ebdfd4a412e4ed4107"} err="failed to get container status \"7d5196071024ad0d76407145ccb91c8412a8875760d0f4ebdfd4a412e4ed4107\": rpc error: code = NotFound desc = could not find container \"7d5196071024ad0d76407145ccb91c8412a8875760d0f4ebdfd4a412e4ed4107\": container with ID starting with 7d5196071024ad0d76407145ccb91c8412a8875760d0f4ebdfd4a412e4ed4107 not found: ID does not exist"
Jul 09 15:31:30.980979 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:30.980948    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp"]
Jul 09 15:31:30.983816 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:30.983795    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726850-fjrmp"]
Jul 09 15:31:31.096596 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:31.096559    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:31:32.231044 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:31:32.231009    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e99604cb-ab23-47e1-a923-9a19efbbe549" path="/var/lib/kubelet/pods/e99604cb-ab23-47e1-a923-9a19efbbe549/volumes"
Jul 09 15:32:30.214171 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:32:30.214142    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:32:30.217478 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:32:30.217450    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:32:53.065227 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:32:53.065193    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:33:00.759748 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:00.759680    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:33:09.967954 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:09.967919    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:33:11.016834 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.016798    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"]
Jul 09 15:33:11.017194 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.017148    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e99604cb-ab23-47e1-a923-9a19efbbe549" containerName="cleanup"
Jul 09 15:33:11.017194 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.017159    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e99604cb-ab23-47e1-a923-9a19efbbe549" containerName="cleanup"
Jul 09 15:33:11.017194 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.017166    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3a33e97-4296-430c-813f-79e8249fd27c" containerName="maas-api"
Jul 09 15:33:11.017194 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.017172    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3a33e97-4296-430c-813f-79e8249fd27c" containerName="maas-api"
Jul 09 15:33:11.017317 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.017224    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e99604cb-ab23-47e1-a923-9a19efbbe549" containerName="cleanup"
Jul 09 15:33:11.017317 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.017232    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3a33e97-4296-430c-813f-79e8249fd27c" containerName="maas-api"
Jul 09 15:33:11.020018 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.020001    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"
Jul 09 15:33:11.024955 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.024934    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-cd893609-serving-cert\""
Jul 09 15:33:11.052927 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.052897    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"]
Jul 09 15:33:11.062167 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.062136    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7shg\" (UniqueName: \"kubernetes.io/projected/1dfd12e3-7295-4112-b0c8-d7608900347f-kube-api-access-d7shg\") pod \"maas-api-e2e-mt-cd893609-5dd9947fd7-5229g\" (UID: \"1dfd12e3-7295-4112-b0c8-d7608900347f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"
Jul 09 15:33:11.062310 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.062239    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dfd12e3-7295-4112-b0c8-d7608900347f-maas-api-tls\") pod \"maas-api-e2e-mt-cd893609-5dd9947fd7-5229g\" (UID: \"1dfd12e3-7295-4112-b0c8-d7608900347f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"
Jul 09 15:33:11.062310 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.062289    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dfd12e3-7295-4112-b0c8-d7608900347f-openshift-service-ca\") pod \"maas-api-e2e-mt-cd893609-5dd9947fd7-5229g\" (UID: \"1dfd12e3-7295-4112-b0c8-d7608900347f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"
Jul 09 15:33:11.163716 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.163657    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dfd12e3-7295-4112-b0c8-d7608900347f-maas-api-tls\") pod \"maas-api-e2e-mt-cd893609-5dd9947fd7-5229g\" (UID: \"1dfd12e3-7295-4112-b0c8-d7608900347f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"
Jul 09 15:33:11.163910 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.163743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dfd12e3-7295-4112-b0c8-d7608900347f-openshift-service-ca\") pod \"maas-api-e2e-mt-cd893609-5dd9947fd7-5229g\" (UID: \"1dfd12e3-7295-4112-b0c8-d7608900347f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"
Jul 09 15:33:11.163910 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.163795    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7shg\" (UniqueName: \"kubernetes.io/projected/1dfd12e3-7295-4112-b0c8-d7608900347f-kube-api-access-d7shg\") pod \"maas-api-e2e-mt-cd893609-5dd9947fd7-5229g\" (UID: \"1dfd12e3-7295-4112-b0c8-d7608900347f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"
Jul 09 15:33:11.164451 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.164424    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dfd12e3-7295-4112-b0c8-d7608900347f-openshift-service-ca\") pod \"maas-api-e2e-mt-cd893609-5dd9947fd7-5229g\" (UID: \"1dfd12e3-7295-4112-b0c8-d7608900347f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"
Jul 09 15:33:11.166174 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.166156    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dfd12e3-7295-4112-b0c8-d7608900347f-maas-api-tls\") pod \"maas-api-e2e-mt-cd893609-5dd9947fd7-5229g\" (UID: \"1dfd12e3-7295-4112-b0c8-d7608900347f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"
Jul 09 15:33:11.183457 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.183434    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7shg\" (UniqueName: \"kubernetes.io/projected/1dfd12e3-7295-4112-b0c8-d7608900347f-kube-api-access-d7shg\") pod \"maas-api-e2e-mt-cd893609-5dd9947fd7-5229g\" (UID: \"1dfd12e3-7295-4112-b0c8-d7608900347f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"
Jul 09 15:33:11.329883 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.329793    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"
Jul 09 15:33:11.474287 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:11.474264    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"]
Jul 09 15:33:11.476445 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:33:11.476416    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dfd12e3_7295_4112_b0c8_d7608900347f.slice/crio-117853b49466c0e10242170812be4c5767913713e20c873d1c61bea072e6fe9e WatchSource:0}: Error finding container 117853b49466c0e10242170812be4c5767913713e20c873d1c61bea072e6fe9e: Status 404 returned error can't find the container with id 117853b49466c0e10242170812be4c5767913713e20c873d1c61bea072e6fe9e
Jul 09 15:33:12.440868 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:12.440834    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g" event={"ID":"1dfd12e3-7295-4112-b0c8-d7608900347f","Type":"ContainerStarted","Data":"117853b49466c0e10242170812be4c5767913713e20c873d1c61bea072e6fe9e"}
Jul 09 15:33:13.453942 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:13.453909    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g" event={"ID":"1dfd12e3-7295-4112-b0c8-d7608900347f","Type":"ContainerStarted","Data":"de7e089822de5bcbfa94a29af680eed8277bbf4a7cc73ef0de77d6a4178d7e99"}
Jul 09 15:33:13.454332 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:13.454061    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"
Jul 09 15:33:13.485553 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:13.485504    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g" podStartSLOduration=1.8733983090000002 podStartE2EDuration="3.485489579s" totalImagesPullingTime="1.61209127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:33:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:33:13.484928048 +0000 UTC m=+1843.818358017" watchObservedRunningTime="2026-07-09 15:33:13.485489579 +0000 UTC m=+1843.818919570"
Jul 09 15:33:19.462817 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:19.462786    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"
Jul 09 15:33:20.125803 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:20.125766    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:33:25.315744 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:25.315706    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"]
Jul 09 15:33:25.316297 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:25.315962    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g" podUID="1dfd12e3-7295-4112-b0c8-d7608900347f" containerName="maas-api" containerID="cri-o://de7e089822de5bcbfa94a29af680eed8277bbf4a7cc73ef0de77d6a4178d7e99" gracePeriod=30
Jul 09 15:33:25.510377 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:25.510253    2579 generic.go:356] "Generic (PLEG): container finished" podID="1dfd12e3-7295-4112-b0c8-d7608900347f" containerID="de7e089822de5bcbfa94a29af680eed8277bbf4a7cc73ef0de77d6a4178d7e99" exitCode=0
Jul 09 15:33:25.510377 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:25.510302    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g" event={"ID":"1dfd12e3-7295-4112-b0c8-d7608900347f","Type":"ContainerDied","Data":"de7e089822de5bcbfa94a29af680eed8277bbf4a7cc73ef0de77d6a4178d7e99"}
Jul 09 15:33:25.577413 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:25.577357    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"
Jul 09 15:33:25.693133 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:25.693097    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7shg\" (UniqueName: \"kubernetes.io/projected/1dfd12e3-7295-4112-b0c8-d7608900347f-kube-api-access-d7shg\") pod \"1dfd12e3-7295-4112-b0c8-d7608900347f\" (UID: \"1dfd12e3-7295-4112-b0c8-d7608900347f\") "
Jul 09 15:33:25.693330 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:25.693154    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dfd12e3-7295-4112-b0c8-d7608900347f-openshift-service-ca\") pod \"1dfd12e3-7295-4112-b0c8-d7608900347f\" (UID: \"1dfd12e3-7295-4112-b0c8-d7608900347f\") "
Jul 09 15:33:25.693330 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:25.693190    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dfd12e3-7295-4112-b0c8-d7608900347f-maas-api-tls\") pod \"1dfd12e3-7295-4112-b0c8-d7608900347f\" (UID: \"1dfd12e3-7295-4112-b0c8-d7608900347f\") "
Jul 09 15:33:25.693528 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:25.693503    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dfd12e3-7295-4112-b0c8-d7608900347f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1dfd12e3-7295-4112-b0c8-d7608900347f" (UID: "1dfd12e3-7295-4112-b0c8-d7608900347f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:33:25.695262 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:25.695234    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dfd12e3-7295-4112-b0c8-d7608900347f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1dfd12e3-7295-4112-b0c8-d7608900347f" (UID: "1dfd12e3-7295-4112-b0c8-d7608900347f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:33:25.695369 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:25.695297    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dfd12e3-7295-4112-b0c8-d7608900347f-kube-api-access-d7shg" (OuterVolumeSpecName: "kube-api-access-d7shg") pod "1dfd12e3-7295-4112-b0c8-d7608900347f" (UID: "1dfd12e3-7295-4112-b0c8-d7608900347f"). InnerVolumeSpecName "kube-api-access-d7shg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:33:25.793919 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:25.793877    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dfd12e3-7295-4112-b0c8-d7608900347f-maas-api-tls\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:33:25.793919 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:25.793911    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7shg\" (UniqueName: \"kubernetes.io/projected/1dfd12e3-7295-4112-b0c8-d7608900347f-kube-api-access-d7shg\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:33:25.793919 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:25.793923    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dfd12e3-7295-4112-b0c8-d7608900347f-openshift-service-ca\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:33:26.516182 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:26.516093    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g" event={"ID":"1dfd12e3-7295-4112-b0c8-d7608900347f","Type":"ContainerDied","Data":"117853b49466c0e10242170812be4c5767913713e20c873d1c61bea072e6fe9e"}
Jul 09 15:33:26.516182 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:26.516128    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"
Jul 09 15:33:26.516182 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:26.516156    2579 scope.go:117] "RemoveContainer" containerID="de7e089822de5bcbfa94a29af680eed8277bbf4a7cc73ef0de77d6a4178d7e99"
Jul 09 15:33:26.535065 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:26.535044    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"]
Jul 09 15:33:26.545059 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:26.545034    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cd893609-5dd9947fd7-5229g"]
Jul 09 15:33:28.230692 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:28.230645    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dfd12e3-7295-4112-b0c8-d7608900347f" path="/var/lib/kubelet/pods/1dfd12e3-7295-4112-b0c8-d7608900347f/volumes"
Jul 09 15:33:29.402923 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:29.402890    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:33:59.990197 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:59.990122    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"]
Jul 09 15:33:59.992333 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:59.990466    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dfd12e3-7295-4112-b0c8-d7608900347f" containerName="maas-api"
Jul 09 15:33:59.992333 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:59.990477    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dfd12e3-7295-4112-b0c8-d7608900347f" containerName="maas-api"
Jul 09 15:33:59.992333 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:59.990553    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dfd12e3-7295-4112-b0c8-d7608900347f" containerName="maas-api"
Jul 09 15:33:59.993167 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:59.993150    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:33:59.997906 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:33:59.997886    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7cc32d82-serving-cert\""
Jul 09 15:34:00.004181 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:00.004158    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"]
Jul 09 15:34:00.096250 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:00.096217    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da84227c-d905-429e-bba0-e19ed1779cda-openshift-service-ca\") pod \"maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld\" (UID: \"da84227c-d905-429e-bba0-e19ed1779cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:34:00.096250 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:00.096256    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da84227c-d905-429e-bba0-e19ed1779cda-maas-api-tls\") pod \"maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld\" (UID: \"da84227c-d905-429e-bba0-e19ed1779cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:34:00.096457 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:00.096279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jmj4\" (UniqueName: \"kubernetes.io/projected/da84227c-d905-429e-bba0-e19ed1779cda-kube-api-access-9jmj4\") pod \"maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld\" (UID: \"da84227c-d905-429e-bba0-e19ed1779cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:34:00.196833 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:00.196793    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da84227c-d905-429e-bba0-e19ed1779cda-openshift-service-ca\") pod \"maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld\" (UID: \"da84227c-d905-429e-bba0-e19ed1779cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:34:00.197002 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:00.196848    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da84227c-d905-429e-bba0-e19ed1779cda-maas-api-tls\") pod \"maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld\" (UID: \"da84227c-d905-429e-bba0-e19ed1779cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:34:00.197002 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:00.196887    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jmj4\" (UniqueName: \"kubernetes.io/projected/da84227c-d905-429e-bba0-e19ed1779cda-kube-api-access-9jmj4\") pod \"maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld\" (UID: \"da84227c-d905-429e-bba0-e19ed1779cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:34:00.197002 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:34:00.196982    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-serving-cert: secret "maas-api-e2e-mt-7cc32d82-serving-cert" not found
Jul 09 15:34:00.197106 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:34:00.197070    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da84227c-d905-429e-bba0-e19ed1779cda-maas-api-tls podName:da84227c-d905-429e-bba0-e19ed1779cda nodeName:}" failed. No retries permitted until 2026-07-09 15:34:00.697050468 +0000 UTC m=+1891.030480411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/da84227c-d905-429e-bba0-e19ed1779cda-maas-api-tls") pod "maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld" (UID: "da84227c-d905-429e-bba0-e19ed1779cda") : secret "maas-api-e2e-mt-7cc32d82-serving-cert" not found
Jul 09 15:34:00.197548 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:00.197531    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da84227c-d905-429e-bba0-e19ed1779cda-openshift-service-ca\") pod \"maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld\" (UID: \"da84227c-d905-429e-bba0-e19ed1779cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:34:00.208154 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:00.208116    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jmj4\" (UniqueName: \"kubernetes.io/projected/da84227c-d905-429e-bba0-e19ed1779cda-kube-api-access-9jmj4\") pod \"maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld\" (UID: \"da84227c-d905-429e-bba0-e19ed1779cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:34:00.701479 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:00.701440    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da84227c-d905-429e-bba0-e19ed1779cda-maas-api-tls\") pod \"maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld\" (UID: \"da84227c-d905-429e-bba0-e19ed1779cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:34:00.703827 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:00.703806    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da84227c-d905-429e-bba0-e19ed1779cda-maas-api-tls\") pod \"maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld\" (UID: \"da84227c-d905-429e-bba0-e19ed1779cda\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:34:00.903409 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:00.903372    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:34:01.067837 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:34:01.067807    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda84227c_d905_429e_bba0_e19ed1779cda.slice/crio-ff59e954a969e121a734e958789b4d144716ec9a7db8a312dbd1efee51a4ad83 WatchSource:0}: Error finding container ff59e954a969e121a734e958789b4d144716ec9a7db8a312dbd1efee51a4ad83: Status 404 returned error can't find the container with id ff59e954a969e121a734e958789b4d144716ec9a7db8a312dbd1efee51a4ad83
Jul 09 15:34:01.068186 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:01.068089    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"]
Jul 09 15:34:01.681290 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:01.681250    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld" event={"ID":"da84227c-d905-429e-bba0-e19ed1779cda","Type":"ContainerStarted","Data":"fd483026cfbe1dbeaf4cf3c5beefad961e7eb15e38b313429c129655944bf7c7"}
Jul 09 15:34:01.681470 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:01.681332    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld" event={"ID":"da84227c-d905-429e-bba0-e19ed1779cda","Type":"ContainerStarted","Data":"ff59e954a969e121a734e958789b4d144716ec9a7db8a312dbd1efee51a4ad83"}
Jul 09 15:34:01.681470 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:01.681394    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:34:01.711192 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:01.711118    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld" podStartSLOduration=2.238202389 podStartE2EDuration="2.711102431s" totalImagesPullingTime="472.900042ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:34:01.710128779 +0000 UTC m=+1892.043558745" watchObservedRunningTime="2026-07-09 15:34:01.711102431 +0000 UTC m=+1892.044532425"
Jul 09 15:34:04.462753 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:04.462712    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:34:07.691869 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:07.691834    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:34:22.634833 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.634799    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"]
Jul 09 15:34:22.638123 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.638101    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.648446 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.648428    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7cc32d82\"/\"e2e-lifecycle-model-7cc32d82-kserve-self-signed-certs\""
Jul 09 15:34:22.648540 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.648443    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7cc32d82\"/\"kube-root-ca.crt\""
Jul 09 15:34:22.648540 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.648434    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7cc32d82\"/\"openshift-service-ca.crt\""
Jul 09 15:34:22.648768 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.648755    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7cc32d82\"/\"default-dockercfg-fmfg5\""
Jul 09 15:34:22.671918 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.671888    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"]
Jul 09 15:34:22.703323 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.703298    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-tls-certs\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.703470 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.703342    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-tmp-dir\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.703470 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.703371    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4q4j\" (UniqueName: \"kubernetes.io/projected/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-kube-api-access-w4q4j\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.703470 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.703434    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-home\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.703584 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.703473    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-model-cache\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.703584 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.703508    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-dshm\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.804556 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.804523    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-model-cache\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.804736 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.804563    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-dshm\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.804736 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.804709    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-tls-certs\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.804845 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.804797    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-tmp-dir\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.804903 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.804850    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4q4j\" (UniqueName: \"kubernetes.io/projected/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-kube-api-access-w4q4j\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.804903 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.804878    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-home\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.805046 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.805021    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-model-cache\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.805155 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.805135    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-tmp-dir\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.805738 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.805714    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-home\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.806809 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.806790    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-dshm\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.807372 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.807356    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-tls-certs\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.817309 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.817286    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4q4j\" (UniqueName: \"kubernetes.io/projected/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-kube-api-access-w4q4j\") pod \"e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") " pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:22.947143 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:22.947104    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:23.105008 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:23.104934    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"]
Jul 09 15:34:23.108135 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:34:23.108109    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9531ded4_6bf6_45fc_adc2_5af6c03cbe89.slice/crio-0a529e808d909b216e6a5a8d7801a52e982a807ed6c67b65816168e0cb987082 WatchSource:0}: Error finding container 0a529e808d909b216e6a5a8d7801a52e982a807ed6c67b65816168e0cb987082: Status 404 returned error can't find the container with id 0a529e808d909b216e6a5a8d7801a52e982a807ed6c67b65816168e0cb987082
Jul 09 15:34:23.787054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:23.787019    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr" event={"ID":"9531ded4-6bf6-45fc-adc2-5af6c03cbe89","Type":"ContainerStarted","Data":"ebc0a8e014408d59059360bd0be47e2228e876d1b471abd14e1621cbc687aadb"}
Jul 09 15:34:23.787054 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:23.787060    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr" event={"ID":"9531ded4-6bf6-45fc-adc2-5af6c03cbe89","Type":"ContainerStarted","Data":"0a529e808d909b216e6a5a8d7801a52e982a807ed6c67b65816168e0cb987082"}
Jul 09 15:34:23.821087 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:23.821038    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr" podStartSLOduration=1.8210219319999998 podStartE2EDuration="1.821021932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:34:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:34:23.820720893 +0000 UTC m=+1914.154150855" watchObservedRunningTime="2026-07-09 15:34:23.821021932 +0000 UTC m=+1914.154451898"
Jul 09 15:34:25.914968 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:25.914934    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:34:32.947622 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:32.947589    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:32.948132 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:32.947648    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:32.960210 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:32.960186    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:33.846450 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:33.846411    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:34:39.509003 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:39.508968    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:34:42.849482 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:42.849451    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"]
Jul 09 15:34:42.849972 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:42.849634    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld" podUID="da84227c-d905-429e-bba0-e19ed1779cda" containerName="maas-api" containerID="cri-o://fd483026cfbe1dbeaf4cf3c5beefad961e7eb15e38b313429c129655944bf7c7" gracePeriod=30
Jul 09 15:34:43.102056 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.101994    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:34:43.180837 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.180806    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jmj4\" (UniqueName: \"kubernetes.io/projected/da84227c-d905-429e-bba0-e19ed1779cda-kube-api-access-9jmj4\") pod \"da84227c-d905-429e-bba0-e19ed1779cda\" (UID: \"da84227c-d905-429e-bba0-e19ed1779cda\") "
Jul 09 15:34:43.181007 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.180880    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da84227c-d905-429e-bba0-e19ed1779cda-maas-api-tls\") pod \"da84227c-d905-429e-bba0-e19ed1779cda\" (UID: \"da84227c-d905-429e-bba0-e19ed1779cda\") "
Jul 09 15:34:43.181007 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.180926    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da84227c-d905-429e-bba0-e19ed1779cda-openshift-service-ca\") pod \"da84227c-d905-429e-bba0-e19ed1779cda\" (UID: \"da84227c-d905-429e-bba0-e19ed1779cda\") "
Jul 09 15:34:43.181293 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.181267    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da84227c-d905-429e-bba0-e19ed1779cda-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "da84227c-d905-429e-bba0-e19ed1779cda" (UID: "da84227c-d905-429e-bba0-e19ed1779cda"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:34:43.182709 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.182658    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da84227c-d905-429e-bba0-e19ed1779cda-kube-api-access-9jmj4" (OuterVolumeSpecName: "kube-api-access-9jmj4") pod "da84227c-d905-429e-bba0-e19ed1779cda" (UID: "da84227c-d905-429e-bba0-e19ed1779cda"). InnerVolumeSpecName "kube-api-access-9jmj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:34:43.182816 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.182759    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da84227c-d905-429e-bba0-e19ed1779cda-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "da84227c-d905-429e-bba0-e19ed1779cda" (UID: "da84227c-d905-429e-bba0-e19ed1779cda"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:34:43.282144 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.282116    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jmj4\" (UniqueName: \"kubernetes.io/projected/da84227c-d905-429e-bba0-e19ed1779cda-kube-api-access-9jmj4\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:34:43.282144 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.282141    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da84227c-d905-429e-bba0-e19ed1779cda-maas-api-tls\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:34:43.282144 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.282150    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da84227c-d905-429e-bba0-e19ed1779cda-openshift-service-ca\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:34:43.890128 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.890096    2579 generic.go:356] "Generic (PLEG): container finished" podID="da84227c-d905-429e-bba0-e19ed1779cda" containerID="fd483026cfbe1dbeaf4cf3c5beefad961e7eb15e38b313429c129655944bf7c7" exitCode=0
Jul 09 15:34:43.890538 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.890157    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"
Jul 09 15:34:43.890538 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.890174    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld" event={"ID":"da84227c-d905-429e-bba0-e19ed1779cda","Type":"ContainerDied","Data":"fd483026cfbe1dbeaf4cf3c5beefad961e7eb15e38b313429c129655944bf7c7"}
Jul 09 15:34:43.890538 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.890270    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld" event={"ID":"da84227c-d905-429e-bba0-e19ed1779cda","Type":"ContainerDied","Data":"ff59e954a969e121a734e958789b4d144716ec9a7db8a312dbd1efee51a4ad83"}
Jul 09 15:34:43.890538 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.890294    2579 scope.go:117] "RemoveContainer" containerID="fd483026cfbe1dbeaf4cf3c5beefad961e7eb15e38b313429c129655944bf7c7"
Jul 09 15:34:43.900051 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.900036    2579 scope.go:117] "RemoveContainer" containerID="fd483026cfbe1dbeaf4cf3c5beefad961e7eb15e38b313429c129655944bf7c7"
Jul 09 15:34:43.900303 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:34:43.900284    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd483026cfbe1dbeaf4cf3c5beefad961e7eb15e38b313429c129655944bf7c7\": container with ID starting with fd483026cfbe1dbeaf4cf3c5beefad961e7eb15e38b313429c129655944bf7c7 not found: ID does not exist" containerID="fd483026cfbe1dbeaf4cf3c5beefad961e7eb15e38b313429c129655944bf7c7"
Jul 09 15:34:43.900349 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.900310    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd483026cfbe1dbeaf4cf3c5beefad961e7eb15e38b313429c129655944bf7c7"} err="failed to get container status \"fd483026cfbe1dbeaf4cf3c5beefad961e7eb15e38b313429c129655944bf7c7\": rpc error: code = NotFound desc = could not find container \"fd483026cfbe1dbeaf4cf3c5beefad961e7eb15e38b313429c129655944bf7c7\": container with ID starting with fd483026cfbe1dbeaf4cf3c5beefad961e7eb15e38b313429c129655944bf7c7 not found: ID does not exist"
Jul 09 15:34:43.916841 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.916819    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"]
Jul 09 15:34:43.922559 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:43.922537    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7cc32d82-6d9cf8b5c7-7qhld"]
Jul 09 15:34:44.231189 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:44.231153    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da84227c-d905-429e-bba0-e19ed1779cda" path="/var/lib/kubelet/pods/da84227c-d905-429e-bba0-e19ed1779cda/volumes"
Jul 09 15:34:53.165438 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:53.165400    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"]
Jul 09 15:34:53.969870 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:34:53.969836    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:35:08.185443 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.185381    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr" podUID="9531ded4-6bf6-45fc-adc2-5af6c03cbe89" containerName="main" containerID="cri-o://ebc0a8e014408d59059360bd0be47e2228e876d1b471abd14e1621cbc687aadb" gracePeriod=45
Jul 09 15:35:08.440291 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.440229    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:35:08.512382 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.512352    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-dshm\") pod \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") "
Jul 09 15:35:08.512541 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.512435    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-home\") pod \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") "
Jul 09 15:35:08.512541 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.512464    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-model-cache\") pod \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") "
Jul 09 15:35:08.512541 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.512496    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-tmp-dir\") pod \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") "
Jul 09 15:35:08.512679 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.512543    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-tls-certs\") pod \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") "
Jul 09 15:35:08.512679 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.512581    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4q4j\" (UniqueName: \"kubernetes.io/projected/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-kube-api-access-w4q4j\") pod \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\" (UID: \"9531ded4-6bf6-45fc-adc2-5af6c03cbe89\") "
Jul 09 15:35:08.512818 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.512752    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-home" (OuterVolumeSpecName: "home") pod "9531ded4-6bf6-45fc-adc2-5af6c03cbe89" (UID: "9531ded4-6bf6-45fc-adc2-5af6c03cbe89"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:35:08.512818 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.512775    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-model-cache" (OuterVolumeSpecName: "model-cache") pod "9531ded4-6bf6-45fc-adc2-5af6c03cbe89" (UID: "9531ded4-6bf6-45fc-adc2-5af6c03cbe89"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:35:08.512931 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.512829    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9531ded4-6bf6-45fc-adc2-5af6c03cbe89" (UID: "9531ded4-6bf6-45fc-adc2-5af6c03cbe89"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:35:08.512931 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.512888    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-home\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:35:08.512931 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.512907    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-model-cache\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:35:08.514645 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.514611    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9531ded4-6bf6-45fc-adc2-5af6c03cbe89" (UID: "9531ded4-6bf6-45fc-adc2-5af6c03cbe89"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:35:08.514804 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.514645    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-dshm" (OuterVolumeSpecName: "dshm") pod "9531ded4-6bf6-45fc-adc2-5af6c03cbe89" (UID: "9531ded4-6bf6-45fc-adc2-5af6c03cbe89"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:35:08.514804 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.514708    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-kube-api-access-w4q4j" (OuterVolumeSpecName: "kube-api-access-w4q4j") pod "9531ded4-6bf6-45fc-adc2-5af6c03cbe89" (UID: "9531ded4-6bf6-45fc-adc2-5af6c03cbe89"). InnerVolumeSpecName "kube-api-access-w4q4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:35:08.613291 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.613256    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-dshm\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:35:08.613291 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.613285    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-tmp-dir\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:35:08.613505 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.613309    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-tls-certs\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:35:08.613505 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:08.613318    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4q4j\" (UniqueName: \"kubernetes.io/projected/9531ded4-6bf6-45fc-adc2-5af6c03cbe89-kube-api-access-w4q4j\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:35:09.014333 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:09.014295    2579 generic.go:356] "Generic (PLEG): container finished" podID="9531ded4-6bf6-45fc-adc2-5af6c03cbe89" containerID="ebc0a8e014408d59059360bd0be47e2228e876d1b471abd14e1621cbc687aadb" exitCode=0
Jul 09 15:35:09.014505 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:09.014336    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"
Jul 09 15:35:09.014505 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:09.014377    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr" event={"ID":"9531ded4-6bf6-45fc-adc2-5af6c03cbe89","Type":"ContainerDied","Data":"ebc0a8e014408d59059360bd0be47e2228e876d1b471abd14e1621cbc687aadb"}
Jul 09 15:35:09.014505 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:09.014422    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr" event={"ID":"9531ded4-6bf6-45fc-adc2-5af6c03cbe89","Type":"ContainerDied","Data":"0a529e808d909b216e6a5a8d7801a52e982a807ed6c67b65816168e0cb987082"}
Jul 09 15:35:09.014505 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:09.014443    2579 scope.go:117] "RemoveContainer" containerID="ebc0a8e014408d59059360bd0be47e2228e876d1b471abd14e1621cbc687aadb"
Jul 09 15:35:09.023622 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:09.023605    2579 scope.go:117] "RemoveContainer" containerID="ebc0a8e014408d59059360bd0be47e2228e876d1b471abd14e1621cbc687aadb"
Jul 09 15:35:09.023914 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:35:09.023893    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebc0a8e014408d59059360bd0be47e2228e876d1b471abd14e1621cbc687aadb\": container with ID starting with ebc0a8e014408d59059360bd0be47e2228e876d1b471abd14e1621cbc687aadb not found: ID does not exist" containerID="ebc0a8e014408d59059360bd0be47e2228e876d1b471abd14e1621cbc687aadb"
Jul 09 15:35:09.023974 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:09.023934    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebc0a8e014408d59059360bd0be47e2228e876d1b471abd14e1621cbc687aadb"} err="failed to get container status \"ebc0a8e014408d59059360bd0be47e2228e876d1b471abd14e1621cbc687aadb\": rpc error: code = NotFound desc = could not find container \"ebc0a8e014408d59059360bd0be47e2228e876d1b471abd14e1621cbc687aadb\": container with ID starting with ebc0a8e014408d59059360bd0be47e2228e876d1b471abd14e1621cbc687aadb not found: ID does not exist"
Jul 09 15:35:09.039500 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:09.039478    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"]
Jul 09 15:35:09.046638 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:09.046617    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-7cc32d82/e2e-lifecycle-model-7cc32d82-kserve-674fbdfc9f-dlvkr"]
Jul 09 15:35:10.231473 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:10.231437    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9531ded4-6bf6-45fc-adc2-5af6c03cbe89" path="/var/lib/kubelet/pods/9531ded4-6bf6-45fc-adc2-5af6c03cbe89/volumes"
Jul 09 15:35:33.060553 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:33.060518    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:35:42.801925 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:42.801877    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:35:55.771316 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:35:55.771278    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:36:37.530569 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.530534    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"]
Jul 09 15:36:37.531011 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.530904    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da84227c-d905-429e-bba0-e19ed1779cda" containerName="maas-api"
Jul 09 15:36:37.531011 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.530917    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="da84227c-d905-429e-bba0-e19ed1779cda" containerName="maas-api"
Jul 09 15:36:37.531011 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.530924    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9531ded4-6bf6-45fc-adc2-5af6c03cbe89" containerName="main"
Jul 09 15:36:37.531011 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.530929    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9531ded4-6bf6-45fc-adc2-5af6c03cbe89" containerName="main"
Jul 09 15:36:37.531011 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.530984    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9531ded4-6bf6-45fc-adc2-5af6c03cbe89" containerName="main"
Jul 09 15:36:37.531011 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.530994    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="da84227c-d905-429e-bba0-e19ed1779cda" containerName="maas-api"
Jul 09 15:36:37.533726 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.533708    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"
Jul 09 15:36:37.537185 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.537157    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-dc88a7-serving-cert\""
Jul 09 15:36:37.549049 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.549025    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"]
Jul 09 15:36:37.660311 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.660265    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d4e24db-1dfa-444d-b7dd-ef6091020b20-maas-api-tls\") pod \"maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv\" (UID: \"1d4e24db-1dfa-444d-b7dd-ef6091020b20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"
Jul 09 15:36:37.660495 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.660341    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2vsn\" (UniqueName: \"kubernetes.io/projected/1d4e24db-1dfa-444d-b7dd-ef6091020b20-kube-api-access-r2vsn\") pod \"maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv\" (UID: \"1d4e24db-1dfa-444d-b7dd-ef6091020b20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"
Jul 09 15:36:37.660495 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.660377    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d4e24db-1dfa-444d-b7dd-ef6091020b20-openshift-service-ca\") pod \"maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv\" (UID: \"1d4e24db-1dfa-444d-b7dd-ef6091020b20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"
Jul 09 15:36:37.761291 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.761255    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d4e24db-1dfa-444d-b7dd-ef6091020b20-openshift-service-ca\") pod \"maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv\" (UID: \"1d4e24db-1dfa-444d-b7dd-ef6091020b20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"
Jul 09 15:36:37.761490 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.761316    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d4e24db-1dfa-444d-b7dd-ef6091020b20-maas-api-tls\") pod \"maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv\" (UID: \"1d4e24db-1dfa-444d-b7dd-ef6091020b20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"
Jul 09 15:36:37.761490 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.761358    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2vsn\" (UniqueName: \"kubernetes.io/projected/1d4e24db-1dfa-444d-b7dd-ef6091020b20-kube-api-access-r2vsn\") pod \"maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv\" (UID: \"1d4e24db-1dfa-444d-b7dd-ef6091020b20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"
Jul 09 15:36:37.762027 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.762001    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d4e24db-1dfa-444d-b7dd-ef6091020b20-openshift-service-ca\") pod \"maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv\" (UID: \"1d4e24db-1dfa-444d-b7dd-ef6091020b20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"
Jul 09 15:36:37.763887 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.763861    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d4e24db-1dfa-444d-b7dd-ef6091020b20-maas-api-tls\") pod \"maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv\" (UID: \"1d4e24db-1dfa-444d-b7dd-ef6091020b20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"
Jul 09 15:36:37.772663 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.772641    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2vsn\" (UniqueName: \"kubernetes.io/projected/1d4e24db-1dfa-444d-b7dd-ef6091020b20-kube-api-access-r2vsn\") pod \"maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv\" (UID: \"1d4e24db-1dfa-444d-b7dd-ef6091020b20\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"
Jul 09 15:36:37.843738 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.843609    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"
Jul 09 15:36:37.995021 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.994916    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"]
Jul 09 15:36:37.997262 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:36:37.997231    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d4e24db_1dfa_444d_b7dd_ef6091020b20.slice/crio-b85439f1f5b558b9ea91192d55f9696250d7cb4702db1107314d93bb34170cd8 WatchSource:0}: Error finding container b85439f1f5b558b9ea91192d55f9696250d7cb4702db1107314d93bb34170cd8: Status 404 returned error can't find the container with id b85439f1f5b558b9ea91192d55f9696250d7cb4702db1107314d93bb34170cd8
Jul 09 15:36:37.998787 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:37.998765    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 15:36:38.446591 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:38.446547    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv" event={"ID":"1d4e24db-1dfa-444d-b7dd-ef6091020b20","Type":"ContainerStarted","Data":"b85439f1f5b558b9ea91192d55f9696250d7cb4702db1107314d93bb34170cd8"}
Jul 09 15:36:39.453441 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:39.453401    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv" event={"ID":"1d4e24db-1dfa-444d-b7dd-ef6091020b20","Type":"ContainerStarted","Data":"d6168138dffb0681ac0a76d856290c2c022d0c034b81fd7fa91f404aa2365b2f"}
Jul 09 15:36:39.453858 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:39.453527    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"
Jul 09 15:36:39.490646 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:39.490588    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv" podStartSLOduration=1.8854436589999999 podStartE2EDuration="2.490570728s" totalImagesPullingTime="605.127069ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:36:39.488280433 +0000 UTC m=+2049.821710397" watchObservedRunningTime="2026-07-09 15:36:39.490570728 +0000 UTC m=+2049.824000716"
Jul 09 15:36:45.463881 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:45.463848    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"
Jul 09 15:36:45.648955 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:45.648911    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"]
Jul 09 15:36:45.653132 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:45.653110    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"
Jul 09 15:36:45.655833 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:45.655810    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-d62682-serving-cert\""
Jul 09 15:36:45.666051 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:45.666015    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"]
Jul 09 15:36:45.732015 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:45.731902    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c307ff0e-c5a5-44b4-bdae-02edf2f55021-maas-api-tls\") pod \"maas-api-e2e-inf-b-d62682-66569d599d-9sfkj\" (UID: \"c307ff0e-c5a5-44b4-bdae-02edf2f55021\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"
Jul 09 15:36:45.732015 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:45.731974    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjzxh\" (UniqueName: \"kubernetes.io/projected/c307ff0e-c5a5-44b4-bdae-02edf2f55021-kube-api-access-qjzxh\") pod \"maas-api-e2e-inf-b-d62682-66569d599d-9sfkj\" (UID: \"c307ff0e-c5a5-44b4-bdae-02edf2f55021\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"
Jul 09 15:36:45.732015 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:45.732015    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c307ff0e-c5a5-44b4-bdae-02edf2f55021-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d62682-66569d599d-9sfkj\" (UID: \"c307ff0e-c5a5-44b4-bdae-02edf2f55021\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"
Jul 09 15:36:45.833115 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:45.833079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjzxh\" (UniqueName: \"kubernetes.io/projected/c307ff0e-c5a5-44b4-bdae-02edf2f55021-kube-api-access-qjzxh\") pod \"maas-api-e2e-inf-b-d62682-66569d599d-9sfkj\" (UID: \"c307ff0e-c5a5-44b4-bdae-02edf2f55021\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"
Jul 09 15:36:45.833326 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:45.833131    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c307ff0e-c5a5-44b4-bdae-02edf2f55021-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d62682-66569d599d-9sfkj\" (UID: \"c307ff0e-c5a5-44b4-bdae-02edf2f55021\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"
Jul 09 15:36:45.833326 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:45.833213    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c307ff0e-c5a5-44b4-bdae-02edf2f55021-maas-api-tls\") pod \"maas-api-e2e-inf-b-d62682-66569d599d-9sfkj\" (UID: \"c307ff0e-c5a5-44b4-bdae-02edf2f55021\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"
Jul 09 15:36:45.833916 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:45.833890    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c307ff0e-c5a5-44b4-bdae-02edf2f55021-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d62682-66569d599d-9sfkj\" (UID: \"c307ff0e-c5a5-44b4-bdae-02edf2f55021\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"
Jul 09 15:36:45.835618 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:45.835599    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c307ff0e-c5a5-44b4-bdae-02edf2f55021-maas-api-tls\") pod \"maas-api-e2e-inf-b-d62682-66569d599d-9sfkj\" (UID: \"c307ff0e-c5a5-44b4-bdae-02edf2f55021\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"
Jul 09 15:36:45.849550 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:45.849519    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjzxh\" (UniqueName: \"kubernetes.io/projected/c307ff0e-c5a5-44b4-bdae-02edf2f55021-kube-api-access-qjzxh\") pod \"maas-api-e2e-inf-b-d62682-66569d599d-9sfkj\" (UID: \"c307ff0e-c5a5-44b4-bdae-02edf2f55021\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"
Jul 09 15:36:45.963497 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:45.963452    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"
Jul 09 15:36:46.126630 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:46.126601    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"]
Jul 09 15:36:46.128028 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:36:46.128001    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc307ff0e_c5a5_44b4_bdae_02edf2f55021.slice/crio-68b47d196ea602bb5dab91c92999240ee360eec79e775478fe340077c689215b WatchSource:0}: Error finding container 68b47d196ea602bb5dab91c92999240ee360eec79e775478fe340077c689215b: Status 404 returned error can't find the container with id 68b47d196ea602bb5dab91c92999240ee360eec79e775478fe340077c689215b
Jul 09 15:36:46.492757 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:46.492712    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj" event={"ID":"c307ff0e-c5a5-44b4-bdae-02edf2f55021","Type":"ContainerStarted","Data":"68b47d196ea602bb5dab91c92999240ee360eec79e775478fe340077c689215b"}
Jul 09 15:36:47.061905 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:47.061871    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:36:50.513634 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:50.513538    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj" event={"ID":"c307ff0e-c5a5-44b4-bdae-02edf2f55021","Type":"ContainerStarted","Data":"08732308ee883409580ffa32c15628f8b5ae5a8b854a93921d6eaac996684292"}
Jul 09 15:36:50.514089 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:50.513666    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"
Jul 09 15:36:50.536535 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:50.536482    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj" podStartSLOduration=1.448491269 podStartE2EDuration="5.536464851s" totalImagesPullingTime="4.087973582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:36:50.535715536 +0000 UTC m=+2060.869145494" watchObservedRunningTime="2026-07-09 15:36:50.536464851 +0000 UTC m=+2060.869894816"
Jul 09 15:36:56.523411 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:36:56.523334    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"
Jul 09 15:37:00.588663 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.588622    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"]
Jul 09 15:37:00.592168 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.592147    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.595235 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.595212    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-dc88a7\"/\"test-model-dc88a7-kserve-self-signed-certs\""
Jul 09 15:37:00.596394 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.596376    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-dc88a7\"/\"openshift-service-ca.crt\""
Jul 09 15:37:00.596496 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.596419    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-dc88a7\"/\"kube-root-ca.crt\""
Jul 09 15:37:00.596658 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.596644    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-dc88a7\"/\"default-dockercfg-xqsz6\""
Jul 09 15:37:00.611987 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.611953    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"]
Jul 09 15:37:00.664172 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.664129    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-tmp-dir\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.664364 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.664179    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88p8w\" (UniqueName: \"kubernetes.io/projected/3253a193-a4b0-406f-98aa-d4bdca9417f0-kube-api-access-88p8w\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.664364 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.664236    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3253a193-a4b0-406f-98aa-d4bdca9417f0-tls-certs\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.664364 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.664280    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-model-cache\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.664364 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.664299    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-home\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.664539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.664396    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-dshm\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.765550 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.765516    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3253a193-a4b0-406f-98aa-d4bdca9417f0-tls-certs\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.765764 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.765558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-model-cache\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.765764 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.765576    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-home\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.765764 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.765629    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-dshm\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.765764 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.765667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-tmp-dir\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.765764 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.765717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88p8w\" (UniqueName: \"kubernetes.io/projected/3253a193-a4b0-406f-98aa-d4bdca9417f0-kube-api-access-88p8w\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.766057 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.765996    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-model-cache\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.766057 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.766049    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-home\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.766195 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.766178    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-tmp-dir\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.768030 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.768006    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-dshm\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.768247 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.768227    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3253a193-a4b0-406f-98aa-d4bdca9417f0-tls-certs\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.777071 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.777041    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88p8w\" (UniqueName: \"kubernetes.io/projected/3253a193-a4b0-406f-98aa-d4bdca9417f0-kube-api-access-88p8w\") pod \"test-model-dc88a7-kserve-6d564df5bc-89lwf\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") " pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:00.901828 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:00.901734    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:01.046019 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:01.045994    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"]
Jul 09 15:37:01.046881 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:37:01.046851    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3253a193_a4b0_406f_98aa_d4bdca9417f0.slice/crio-03242483f314c108368e24641533df373d822a5dd68a3baf74dc37f7111acf2a WatchSource:0}: Error finding container 03242483f314c108368e24641533df373d822a5dd68a3baf74dc37f7111acf2a: Status 404 returned error can't find the container with id 03242483f314c108368e24641533df373d822a5dd68a3baf74dc37f7111acf2a
Jul 09 15:37:01.574861 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:01.574821    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf" event={"ID":"3253a193-a4b0-406f-98aa-d4bdca9417f0","Type":"ContainerStarted","Data":"f1e0e02b50fb772e9792d5df1ff308e8eb6896851156941c21327b42806889bb"}
Jul 09 15:37:01.574861 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:01.574868    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf" event={"ID":"3253a193-a4b0-406f-98aa-d4bdca9417f0","Type":"ContainerStarted","Data":"03242483f314c108368e24641533df373d822a5dd68a3baf74dc37f7111acf2a"}
Jul 09 15:37:01.594332 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:01.594280    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf" podStartSLOduration=1.594262988 podStartE2EDuration="1.594262988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:37:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:37:01.593896422 +0000 UTC m=+2071.927326398" watchObservedRunningTime="2026-07-09 15:37:01.594262988 +0000 UTC m=+2071.927693015"
Jul 09 15:37:01.957746 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:01.957706    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:37:10.902557 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:10.902519    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:10.903105 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:10.902578    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:10.915094 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:10.915060    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:11.635284 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:11.635254    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:37:13.357538 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:13.357494    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:37:27.269181 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.269145    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"]
Jul 09 15:37:27.272814 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.272796    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.276433 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.276410    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d62682\"/\"kube-root-ca.crt\""
Jul 09 15:37:27.276578 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.276555    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d62682\"/\"openshift-service-ca.crt\""
Jul 09 15:37:27.276635 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.276562    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d62682\"/\"default-dockercfg-rjgkk\""
Jul 09 15:37:27.276635 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.276562    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d62682\"/\"test-model-d62682-kserve-self-signed-certs\""
Jul 09 15:37:27.283739 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.283717    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"]
Jul 09 15:37:27.402773 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.402737    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-dshm\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.402773 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.402774    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa750835-46d1-4134-9086-18fb0dd297c5-tls-certs\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.403052 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.402819    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-home\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.403052 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.402848    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhnnb\" (UniqueName: \"kubernetes.io/projected/fa750835-46d1-4134-9086-18fb0dd297c5-kube-api-access-vhnnb\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.403052 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.402967    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-tmp-dir\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.403052 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.403021    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-model-cache\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.503796 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.503760    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa750835-46d1-4134-9086-18fb0dd297c5-tls-certs\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.504004 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.503814    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-home\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.504004 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.503942    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhnnb\" (UniqueName: \"kubernetes.io/projected/fa750835-46d1-4134-9086-18fb0dd297c5-kube-api-access-vhnnb\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.504129 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.504067    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-tmp-dir\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.504186 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.504114    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-model-cache\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.504186 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.504174    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-dshm\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.504284 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.504207    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-home\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.505101 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.504783    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-model-cache\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.505101 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.504885    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-tmp-dir\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.507723 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.507446    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-dshm\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.513604 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.513579    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa750835-46d1-4134-9086-18fb0dd297c5-tls-certs\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.517050 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.517022    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhnnb\" (UniqueName: \"kubernetes.io/projected/fa750835-46d1-4134-9086-18fb0dd297c5-kube-api-access-vhnnb\") pod \"test-model-d62682-kserve-68587b7db5-rslw6\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") " pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.583100 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.583006    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:27.726632 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:27.726606    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"]
Jul 09 15:37:27.728533 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:37:27.728504    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa750835_46d1_4134_9086_18fb0dd297c5.slice/crio-07b58839ec2bcf1d0aa527e7f99d95f6bc266f355e434898b3708851f2b2d390 WatchSource:0}: Error finding container 07b58839ec2bcf1d0aa527e7f99d95f6bc266f355e434898b3708851f2b2d390: Status 404 returned error can't find the container with id 07b58839ec2bcf1d0aa527e7f99d95f6bc266f355e434898b3708851f2b2d390
Jul 09 15:37:28.711849 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:28.711806    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6" event={"ID":"fa750835-46d1-4134-9086-18fb0dd297c5","Type":"ContainerStarted","Data":"855b84d2cf012e00301f9db99d08d5c89946808428e533dc7a1f056a093155b7"}
Jul 09 15:37:28.712318 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:28.711863    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6" event={"ID":"fa750835-46d1-4134-9086-18fb0dd297c5","Type":"ContainerStarted","Data":"07b58839ec2bcf1d0aa527e7f99d95f6bc266f355e434898b3708851f2b2d390"}
Jul 09 15:37:28.743167 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:28.743108    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6" podStartSLOduration=1.743090458 podStartE2EDuration="1.743090458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:37:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:37:28.742940561 +0000 UTC m=+2099.076370528" watchObservedRunningTime="2026-07-09 15:37:28.743090458 +0000 UTC m=+2099.076520421"
Jul 09 15:37:30.244765 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:30.244738    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:37:30.247914 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:30.247890    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:37:31.566927 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:31.566888    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:37:37.583765 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:37.583720    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:37.584250 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:37.583779    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:37.596565 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:37.596536    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:37.769789 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:37.769746    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:37:45.873901 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:45.873861    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:37:48.762091 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:48.762052    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"]
Jul 09 15:37:49.637551 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:49.637510    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"]
Jul 09 15:37:50.684848 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:50.684798    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"]
Jul 09 15:37:50.685329 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:50.685115    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv" podUID="1d4e24db-1dfa-444d-b7dd-ef6091020b20" containerName="maas-api" containerID="cri-o://d6168138dffb0681ac0a76d856290c2c022d0c034b81fd7fa91f404aa2365b2f" gracePeriod=30
Jul 09 15:37:50.825079 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:50.825048    2579 generic.go:356] "Generic (PLEG): container finished" podID="1d4e24db-1dfa-444d-b7dd-ef6091020b20" containerID="d6168138dffb0681ac0a76d856290c2c022d0c034b81fd7fa91f404aa2365b2f" exitCode=0
Jul 09 15:37:50.825240 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:50.825124    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv" event={"ID":"1d4e24db-1dfa-444d-b7dd-ef6091020b20","Type":"ContainerDied","Data":"d6168138dffb0681ac0a76d856290c2c022d0c034b81fd7fa91f404aa2365b2f"}
Jul 09 15:37:50.937222 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:50.937155    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"
Jul 09 15:37:51.114555 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:51.114516    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2vsn\" (UniqueName: \"kubernetes.io/projected/1d4e24db-1dfa-444d-b7dd-ef6091020b20-kube-api-access-r2vsn\") pod \"1d4e24db-1dfa-444d-b7dd-ef6091020b20\" (UID: \"1d4e24db-1dfa-444d-b7dd-ef6091020b20\") "
Jul 09 15:37:51.114555 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:51.114558    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d4e24db-1dfa-444d-b7dd-ef6091020b20-openshift-service-ca\") pod \"1d4e24db-1dfa-444d-b7dd-ef6091020b20\" (UID: \"1d4e24db-1dfa-444d-b7dd-ef6091020b20\") "
Jul 09 15:37:51.114796 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:51.114666    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d4e24db-1dfa-444d-b7dd-ef6091020b20-maas-api-tls\") pod \"1d4e24db-1dfa-444d-b7dd-ef6091020b20\" (UID: \"1d4e24db-1dfa-444d-b7dd-ef6091020b20\") "
Jul 09 15:37:51.115075 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:51.115044    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d4e24db-1dfa-444d-b7dd-ef6091020b20-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1d4e24db-1dfa-444d-b7dd-ef6091020b20" (UID: "1d4e24db-1dfa-444d-b7dd-ef6091020b20"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:37:51.116741 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:51.116711    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d4e24db-1dfa-444d-b7dd-ef6091020b20-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1d4e24db-1dfa-444d-b7dd-ef6091020b20" (UID: "1d4e24db-1dfa-444d-b7dd-ef6091020b20"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:37:51.116852 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:51.116745    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d4e24db-1dfa-444d-b7dd-ef6091020b20-kube-api-access-r2vsn" (OuterVolumeSpecName: "kube-api-access-r2vsn") pod "1d4e24db-1dfa-444d-b7dd-ef6091020b20" (UID: "1d4e24db-1dfa-444d-b7dd-ef6091020b20"). InnerVolumeSpecName "kube-api-access-r2vsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:37:51.215410 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:51.215315    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d4e24db-1dfa-444d-b7dd-ef6091020b20-maas-api-tls\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:37:51.215410 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:51.215351    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2vsn\" (UniqueName: \"kubernetes.io/projected/1d4e24db-1dfa-444d-b7dd-ef6091020b20-kube-api-access-r2vsn\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:37:51.215410 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:51.215365    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d4e24db-1dfa-444d-b7dd-ef6091020b20-openshift-service-ca\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:37:51.830934 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:51.830898    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv" event={"ID":"1d4e24db-1dfa-444d-b7dd-ef6091020b20","Type":"ContainerDied","Data":"b85439f1f5b558b9ea91192d55f9696250d7cb4702db1107314d93bb34170cd8"}
Jul 09 15:37:51.830934 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:51.830938    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"
Jul 09 15:37:51.831446 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:51.830955    2579 scope.go:117] "RemoveContainer" containerID="d6168138dffb0681ac0a76d856290c2c022d0c034b81fd7fa91f404aa2365b2f"
Jul 09 15:37:51.864927 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:51.864904    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"]
Jul 09 15:37:51.871046 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:51.871026    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dc88a7-7899d74f5b-qlfpv"]
Jul 09 15:37:52.231145 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:37:52.231117    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d4e24db-1dfa-444d-b7dd-ef6091020b20" path="/var/lib/kubelet/pods/1d4e24db-1dfa-444d-b7dd-ef6091020b20/volumes"
Jul 09 15:38:03.779541 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:03.779505    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:38:03.782189 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:03.782159    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf" podUID="3253a193-a4b0-406f-98aa-d4bdca9417f0" containerName="main" containerID="cri-o://f1e0e02b50fb772e9792d5df1ff308e8eb6896851156941c21327b42806889bb" gracePeriod=45
Jul 09 15:38:04.028758 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.028733    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:38:04.112675 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.112600    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-model-cache\") pod \"3253a193-a4b0-406f-98aa-d4bdca9417f0\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") "
Jul 09 15:38:04.112675 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.112638    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3253a193-a4b0-406f-98aa-d4bdca9417f0-tls-certs\") pod \"3253a193-a4b0-406f-98aa-d4bdca9417f0\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") "
Jul 09 15:38:04.112675 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.112665    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-home\") pod \"3253a193-a4b0-406f-98aa-d4bdca9417f0\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") "
Jul 09 15:38:04.112915 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.112766    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-tmp-dir\") pod \"3253a193-a4b0-406f-98aa-d4bdca9417f0\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") "
Jul 09 15:38:04.112915 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.112796    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88p8w\" (UniqueName: \"kubernetes.io/projected/3253a193-a4b0-406f-98aa-d4bdca9417f0-kube-api-access-88p8w\") pod \"3253a193-a4b0-406f-98aa-d4bdca9417f0\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") "
Jul 09 15:38:04.112915 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.112841    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-dshm\") pod \"3253a193-a4b0-406f-98aa-d4bdca9417f0\" (UID: \"3253a193-a4b0-406f-98aa-d4bdca9417f0\") "
Jul 09 15:38:04.113076 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.112927    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-home" (OuterVolumeSpecName: "home") pod "3253a193-a4b0-406f-98aa-d4bdca9417f0" (UID: "3253a193-a4b0-406f-98aa-d4bdca9417f0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:38:04.113076 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.112934    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-model-cache" (OuterVolumeSpecName: "model-cache") pod "3253a193-a4b0-406f-98aa-d4bdca9417f0" (UID: "3253a193-a4b0-406f-98aa-d4bdca9417f0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:38:04.113076 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.113037    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3253a193-a4b0-406f-98aa-d4bdca9417f0" (UID: "3253a193-a4b0-406f-98aa-d4bdca9417f0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:38:04.113229 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.113140    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-model-cache\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:38:04.113229 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.113164    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-home\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:38:04.113229 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.113181    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-tmp-dir\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:38:04.114920 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.114894    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-dshm" (OuterVolumeSpecName: "dshm") pod "3253a193-a4b0-406f-98aa-d4bdca9417f0" (UID: "3253a193-a4b0-406f-98aa-d4bdca9417f0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:38:04.115010 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.114915    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3253a193-a4b0-406f-98aa-d4bdca9417f0-kube-api-access-88p8w" (OuterVolumeSpecName: "kube-api-access-88p8w") pod "3253a193-a4b0-406f-98aa-d4bdca9417f0" (UID: "3253a193-a4b0-406f-98aa-d4bdca9417f0"). InnerVolumeSpecName "kube-api-access-88p8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:38:04.115047 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.115020    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3253a193-a4b0-406f-98aa-d4bdca9417f0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3253a193-a4b0-406f-98aa-d4bdca9417f0" (UID: "3253a193-a4b0-406f-98aa-d4bdca9417f0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:38:04.213552 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.213528    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3253a193-a4b0-406f-98aa-d4bdca9417f0-tls-certs\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:38:04.213552 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.213550    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88p8w\" (UniqueName: \"kubernetes.io/projected/3253a193-a4b0-406f-98aa-d4bdca9417f0-kube-api-access-88p8w\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:38:04.213722 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.213560    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3253a193-a4b0-406f-98aa-d4bdca9417f0-dshm\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:38:04.657721 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.657664    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6" podUID="fa750835-46d1-4134-9086-18fb0dd297c5" containerName="main" containerID="cri-o://855b84d2cf012e00301f9db99d08d5c89946808428e533dc7a1f056a093155b7" gracePeriod=45
Jul 09 15:38:04.897453 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.897425    2579 generic.go:356] "Generic (PLEG): container finished" podID="3253a193-a4b0-406f-98aa-d4bdca9417f0" containerID="f1e0e02b50fb772e9792d5df1ff308e8eb6896851156941c21327b42806889bb" exitCode=0
Jul 09 15:38:04.897824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.897470    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"
Jul 09 15:38:04.897824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.897503    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf" event={"ID":"3253a193-a4b0-406f-98aa-d4bdca9417f0","Type":"ContainerDied","Data":"f1e0e02b50fb772e9792d5df1ff308e8eb6896851156941c21327b42806889bb"}
Jul 09 15:38:04.897824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.897547    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf" event={"ID":"3253a193-a4b0-406f-98aa-d4bdca9417f0","Type":"ContainerDied","Data":"03242483f314c108368e24641533df373d822a5dd68a3baf74dc37f7111acf2a"}
Jul 09 15:38:04.897824 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.897570    2579 scope.go:117] "RemoveContainer" containerID="f1e0e02b50fb772e9792d5df1ff308e8eb6896851156941c21327b42806889bb"
Jul 09 15:38:04.899058 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.899039    2579 generic.go:356] "Generic (PLEG): container finished" podID="fa750835-46d1-4134-9086-18fb0dd297c5" containerID="855b84d2cf012e00301f9db99d08d5c89946808428e533dc7a1f056a093155b7" exitCode=0
Jul 09 15:38:04.899156 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.899070    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6" event={"ID":"fa750835-46d1-4134-9086-18fb0dd297c5","Type":"ContainerDied","Data":"855b84d2cf012e00301f9db99d08d5c89946808428e533dc7a1f056a093155b7"}
Jul 09 15:38:04.899156 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.899099    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6" event={"ID":"fa750835-46d1-4134-9086-18fb0dd297c5","Type":"ContainerDied","Data":"07b58839ec2bcf1d0aa527e7f99d95f6bc266f355e434898b3708851f2b2d390"}
Jul 09 15:38:04.899156 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.899115    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07b58839ec2bcf1d0aa527e7f99d95f6bc266f355e434898b3708851f2b2d390"
Jul 09 15:38:04.911057 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.911040    2579 scope.go:117] "RemoveContainer" containerID="f1e0e02b50fb772e9792d5df1ff308e8eb6896851156941c21327b42806889bb"
Jul 09 15:38:04.911294 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:38:04.911277    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1e0e02b50fb772e9792d5df1ff308e8eb6896851156941c21327b42806889bb\": container with ID starting with f1e0e02b50fb772e9792d5df1ff308e8eb6896851156941c21327b42806889bb not found: ID does not exist" containerID="f1e0e02b50fb772e9792d5df1ff308e8eb6896851156941c21327b42806889bb"
Jul 09 15:38:04.911354 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.911307    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1e0e02b50fb772e9792d5df1ff308e8eb6896851156941c21327b42806889bb"} err="failed to get container status \"f1e0e02b50fb772e9792d5df1ff308e8eb6896851156941c21327b42806889bb\": rpc error: code = NotFound desc = could not find container \"f1e0e02b50fb772e9792d5df1ff308e8eb6896851156941c21327b42806889bb\": container with ID starting with f1e0e02b50fb772e9792d5df1ff308e8eb6896851156941c21327b42806889bb not found: ID does not exist"
Jul 09 15:38:04.913017 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.912997    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:38:04.926282 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.926255    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"]
Jul 09 15:38:04.930734 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:04.930716    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-dc88a7/test-model-dc88a7-kserve-6d564df5bc-89lwf"]
Jul 09 15:38:05.020445 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.020393    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-dshm\") pod \"fa750835-46d1-4134-9086-18fb0dd297c5\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") "
Jul 09 15:38:05.020626 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.020470    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhnnb\" (UniqueName: \"kubernetes.io/projected/fa750835-46d1-4134-9086-18fb0dd297c5-kube-api-access-vhnnb\") pod \"fa750835-46d1-4134-9086-18fb0dd297c5\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") "
Jul 09 15:38:05.020626 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.020490    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-model-cache\") pod \"fa750835-46d1-4134-9086-18fb0dd297c5\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") "
Jul 09 15:38:05.020626 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.020559    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-tmp-dir\") pod \"fa750835-46d1-4134-9086-18fb0dd297c5\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") "
Jul 09 15:38:05.020839 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.020677    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa750835-46d1-4134-9086-18fb0dd297c5-tls-certs\") pod \"fa750835-46d1-4134-9086-18fb0dd297c5\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") "
Jul 09 15:38:05.020839 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.020754    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-home\") pod \"fa750835-46d1-4134-9086-18fb0dd297c5\" (UID: \"fa750835-46d1-4134-9086-18fb0dd297c5\") "
Jul 09 15:38:05.020942 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.020853    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fa750835-46d1-4134-9086-18fb0dd297c5" (UID: "fa750835-46d1-4134-9086-18fb0dd297c5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:38:05.020942 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.020879    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-model-cache" (OuterVolumeSpecName: "model-cache") pod "fa750835-46d1-4134-9086-18fb0dd297c5" (UID: "fa750835-46d1-4134-9086-18fb0dd297c5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:38:05.021072 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.021055    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-tmp-dir\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:38:05.021126 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.021066    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-home" (OuterVolumeSpecName: "home") pod "fa750835-46d1-4134-9086-18fb0dd297c5" (UID: "fa750835-46d1-4134-9086-18fb0dd297c5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:38:05.021126 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.021077    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-model-cache\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:38:05.022803 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.022749    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-dshm" (OuterVolumeSpecName: "dshm") pod "fa750835-46d1-4134-9086-18fb0dd297c5" (UID: "fa750835-46d1-4134-9086-18fb0dd297c5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:38:05.022901 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.022826    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa750835-46d1-4134-9086-18fb0dd297c5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fa750835-46d1-4134-9086-18fb0dd297c5" (UID: "fa750835-46d1-4134-9086-18fb0dd297c5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:38:05.022901 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.022855    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa750835-46d1-4134-9086-18fb0dd297c5-kube-api-access-vhnnb" (OuterVolumeSpecName: "kube-api-access-vhnnb") pod "fa750835-46d1-4134-9086-18fb0dd297c5" (UID: "fa750835-46d1-4134-9086-18fb0dd297c5"). InnerVolumeSpecName "kube-api-access-vhnnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:38:05.121553 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.121512    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa750835-46d1-4134-9086-18fb0dd297c5-tls-certs\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:38:05.121553 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.121547    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-home\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:38:05.121553 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.121555    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa750835-46d1-4134-9086-18fb0dd297c5-dshm\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:38:05.121553 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.121564    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhnnb\" (UniqueName: \"kubernetes.io/projected/fa750835-46d1-4134-9086-18fb0dd297c5-kube-api-access-vhnnb\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:38:05.905375 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.905343    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"
Jul 09 15:38:05.928571 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.928536    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"]
Jul 09 15:38:05.934050 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:05.934023    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-d62682/test-model-d62682-kserve-68587b7db5-rslw6"]
Jul 09 15:38:06.231061 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:06.231025    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3253a193-a4b0-406f-98aa-d4bdca9417f0" path="/var/lib/kubelet/pods/3253a193-a4b0-406f-98aa-d4bdca9417f0/volumes"
Jul 09 15:38:06.231438 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:06.231423    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa750835-46d1-4134-9086-18fb0dd297c5" path="/var/lib/kubelet/pods/fa750835-46d1-4134-9086-18fb0dd297c5/volumes"
Jul 09 15:38:32.295624 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:32.295583    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"]
Jul 09 15:38:32.296139 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:32.295863    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj" podUID="c307ff0e-c5a5-44b4-bdae-02edf2f55021" containerName="maas-api" containerID="cri-o://08732308ee883409580ffa32c15628f8b5ae5a8b854a93921d6eaac996684292" gracePeriod=30
Jul 09 15:38:32.552816 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:32.552756    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"
Jul 09 15:38:32.650252 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:32.650216    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c307ff0e-c5a5-44b4-bdae-02edf2f55021-openshift-service-ca\") pod \"c307ff0e-c5a5-44b4-bdae-02edf2f55021\" (UID: \"c307ff0e-c5a5-44b4-bdae-02edf2f55021\") "
Jul 09 15:38:32.650453 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:32.650272    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c307ff0e-c5a5-44b4-bdae-02edf2f55021-maas-api-tls\") pod \"c307ff0e-c5a5-44b4-bdae-02edf2f55021\" (UID: \"c307ff0e-c5a5-44b4-bdae-02edf2f55021\") "
Jul 09 15:38:32.650453 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:32.650294    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjzxh\" (UniqueName: \"kubernetes.io/projected/c307ff0e-c5a5-44b4-bdae-02edf2f55021-kube-api-access-qjzxh\") pod \"c307ff0e-c5a5-44b4-bdae-02edf2f55021\" (UID: \"c307ff0e-c5a5-44b4-bdae-02edf2f55021\") "
Jul 09 15:38:32.650738 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:32.650681    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c307ff0e-c5a5-44b4-bdae-02edf2f55021-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c307ff0e-c5a5-44b4-bdae-02edf2f55021" (UID: "c307ff0e-c5a5-44b4-bdae-02edf2f55021"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:38:32.652380 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:32.652355    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c307ff0e-c5a5-44b4-bdae-02edf2f55021-kube-api-access-qjzxh" (OuterVolumeSpecName: "kube-api-access-qjzxh") pod "c307ff0e-c5a5-44b4-bdae-02edf2f55021" (UID: "c307ff0e-c5a5-44b4-bdae-02edf2f55021"). InnerVolumeSpecName "kube-api-access-qjzxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:38:32.652380 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:32.652369    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c307ff0e-c5a5-44b4-bdae-02edf2f55021-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c307ff0e-c5a5-44b4-bdae-02edf2f55021" (UID: "c307ff0e-c5a5-44b4-bdae-02edf2f55021"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:38:32.750920 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:32.750884    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c307ff0e-c5a5-44b4-bdae-02edf2f55021-maas-api-tls\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:38:32.750920 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:32.750915    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjzxh\" (UniqueName: \"kubernetes.io/projected/c307ff0e-c5a5-44b4-bdae-02edf2f55021-kube-api-access-qjzxh\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:38:32.750920 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:32.750924    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c307ff0e-c5a5-44b4-bdae-02edf2f55021-openshift-service-ca\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:38:33.036140 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:33.036103    2579 generic.go:356] "Generic (PLEG): container finished" podID="c307ff0e-c5a5-44b4-bdae-02edf2f55021" containerID="08732308ee883409580ffa32c15628f8b5ae5a8b854a93921d6eaac996684292" exitCode=0
Jul 09 15:38:33.036314 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:33.036150    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"
Jul 09 15:38:33.036314 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:33.036174    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj" event={"ID":"c307ff0e-c5a5-44b4-bdae-02edf2f55021","Type":"ContainerDied","Data":"08732308ee883409580ffa32c15628f8b5ae5a8b854a93921d6eaac996684292"}
Jul 09 15:38:33.036314 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:33.036215    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj" event={"ID":"c307ff0e-c5a5-44b4-bdae-02edf2f55021","Type":"ContainerDied","Data":"68b47d196ea602bb5dab91c92999240ee360eec79e775478fe340077c689215b"}
Jul 09 15:38:33.036314 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:33.036237    2579 scope.go:117] "RemoveContainer" containerID="08732308ee883409580ffa32c15628f8b5ae5a8b854a93921d6eaac996684292"
Jul 09 15:38:33.046367 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:33.046349    2579 scope.go:117] "RemoveContainer" containerID="08732308ee883409580ffa32c15628f8b5ae5a8b854a93921d6eaac996684292"
Jul 09 15:38:33.046645 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:38:33.046624    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08732308ee883409580ffa32c15628f8b5ae5a8b854a93921d6eaac996684292\": container with ID starting with 08732308ee883409580ffa32c15628f8b5ae5a8b854a93921d6eaac996684292 not found: ID does not exist" containerID="08732308ee883409580ffa32c15628f8b5ae5a8b854a93921d6eaac996684292"
Jul 09 15:38:33.046769 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:33.046654    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08732308ee883409580ffa32c15628f8b5ae5a8b854a93921d6eaac996684292"} err="failed to get container status \"08732308ee883409580ffa32c15628f8b5ae5a8b854a93921d6eaac996684292\": rpc error: code = NotFound desc = could not find container \"08732308ee883409580ffa32c15628f8b5ae5a8b854a93921d6eaac996684292\": container with ID starting with 08732308ee883409580ffa32c15628f8b5ae5a8b854a93921d6eaac996684292 not found: ID does not exist"
Jul 09 15:38:33.064467 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:33.064435    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"]
Jul 09 15:38:33.069228 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:33.069201    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d62682-66569d599d-9sfkj"]
Jul 09 15:38:34.230908 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:34.230871    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c307ff0e-c5a5-44b4-bdae-02edf2f55021" path="/var/lib/kubelet/pods/c307ff0e-c5a5-44b4-bdae-02edf2f55021/volumes"
Jul 09 15:38:35.592346 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:35.592309    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:38:42.409651 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.409613    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"]
Jul 09 15:38:42.410116 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.409946    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa750835-46d1-4134-9086-18fb0dd297c5" containerName="main"
Jul 09 15:38:42.410116 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.409957    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa750835-46d1-4134-9086-18fb0dd297c5" containerName="main"
Jul 09 15:38:42.410116 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.409970    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c307ff0e-c5a5-44b4-bdae-02edf2f55021" containerName="maas-api"
Jul 09 15:38:42.410116 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.409976    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c307ff0e-c5a5-44b4-bdae-02edf2f55021" containerName="maas-api"
Jul 09 15:38:42.410116 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.409985    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4e24db-1dfa-444d-b7dd-ef6091020b20" containerName="maas-api"
Jul 09 15:38:42.410116 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.409991    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4e24db-1dfa-444d-b7dd-ef6091020b20" containerName="maas-api"
Jul 09 15:38:42.410116 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.410000    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3253a193-a4b0-406f-98aa-d4bdca9417f0" containerName="main"
Jul 09 15:38:42.410116 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.410006    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3253a193-a4b0-406f-98aa-d4bdca9417f0" containerName="main"
Jul 09 15:38:42.410116 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.410065    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c307ff0e-c5a5-44b4-bdae-02edf2f55021" containerName="maas-api"
Jul 09 15:38:42.410116 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.410074    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa750835-46d1-4134-9086-18fb0dd297c5" containerName="main"
Jul 09 15:38:42.410116 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.410080    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3253a193-a4b0-406f-98aa-d4bdca9417f0" containerName="main"
Jul 09 15:38:42.410116 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.410088    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4e24db-1dfa-444d-b7dd-ef6091020b20" containerName="maas-api"
Jul 09 15:38:42.413039 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.413022    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:38:42.419513 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.419326    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-0164f1-serving-cert\""
Jul 09 15:38:42.428226 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.428206    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"]
Jul 09 15:38:42.537393 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.537351    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcp75\" (UniqueName: \"kubernetes.io/projected/66a67b53-128c-4670-b31a-08a0a7e997e4-kube-api-access-hcp75\") pod \"maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f\" (UID: \"66a67b53-128c-4670-b31a-08a0a7e997e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:38:42.537618 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.537401    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66a67b53-128c-4670-b31a-08a0a7e997e4-maas-api-tls\") pod \"maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f\" (UID: \"66a67b53-128c-4670-b31a-08a0a7e997e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:38:42.537618 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.537434    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66a67b53-128c-4670-b31a-08a0a7e997e4-openshift-service-ca\") pod \"maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f\" (UID: \"66a67b53-128c-4670-b31a-08a0a7e997e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:38:42.638732 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.638677    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcp75\" (UniqueName: \"kubernetes.io/projected/66a67b53-128c-4670-b31a-08a0a7e997e4-kube-api-access-hcp75\") pod \"maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f\" (UID: \"66a67b53-128c-4670-b31a-08a0a7e997e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:38:42.638880 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.638750    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66a67b53-128c-4670-b31a-08a0a7e997e4-maas-api-tls\") pod \"maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f\" (UID: \"66a67b53-128c-4670-b31a-08a0a7e997e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:38:42.638880 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.638776    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66a67b53-128c-4670-b31a-08a0a7e997e4-openshift-service-ca\") pod \"maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f\" (UID: \"66a67b53-128c-4670-b31a-08a0a7e997e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:38:42.638979 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:38:42.638879    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-serving-cert: secret "maas-api-e2e-api-a-0164f1-serving-cert" not found
Jul 09 15:38:42.638979 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:38:42.638947    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66a67b53-128c-4670-b31a-08a0a7e997e4-maas-api-tls podName:66a67b53-128c-4670-b31a-08a0a7e997e4 nodeName:}" failed. No retries permitted until 2026-07-09 15:38:43.138932189 +0000 UTC m=+2173.472362131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/66a67b53-128c-4670-b31a-08a0a7e997e4-maas-api-tls") pod "maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f" (UID: "66a67b53-128c-4670-b31a-08a0a7e997e4") : secret "maas-api-e2e-api-a-0164f1-serving-cert" not found
Jul 09 15:38:42.639435 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.639418    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66a67b53-128c-4670-b31a-08a0a7e997e4-openshift-service-ca\") pod \"maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f\" (UID: \"66a67b53-128c-4670-b31a-08a0a7e997e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:38:42.662038 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:42.661973    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcp75\" (UniqueName: \"kubernetes.io/projected/66a67b53-128c-4670-b31a-08a0a7e997e4-kube-api-access-hcp75\") pod \"maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f\" (UID: \"66a67b53-128c-4670-b31a-08a0a7e997e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:38:43.144271 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:43.144237    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66a67b53-128c-4670-b31a-08a0a7e997e4-maas-api-tls\") pod \"maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f\" (UID: \"66a67b53-128c-4670-b31a-08a0a7e997e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:38:43.146769 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:43.146742    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66a67b53-128c-4670-b31a-08a0a7e997e4-maas-api-tls\") pod \"maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f\" (UID: \"66a67b53-128c-4670-b31a-08a0a7e997e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:38:43.322735 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:43.322678    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:38:43.468726 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:43.468681    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"]
Jul 09 15:38:43.472364 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:38:43.472332    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66a67b53_128c_4670_b31a_08a0a7e997e4.slice/crio-2825f2e0edeb48c81e9c161da2c75b037d0f0736ec9853d7d43e7a1d525045e2 WatchSource:0}: Error finding container 2825f2e0edeb48c81e9c161da2c75b037d0f0736ec9853d7d43e7a1d525045e2: Status 404 returned error can't find the container with id 2825f2e0edeb48c81e9c161da2c75b037d0f0736ec9853d7d43e7a1d525045e2
Jul 09 15:38:44.093913 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:44.093883    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f" event={"ID":"66a67b53-128c-4670-b31a-08a0a7e997e4","Type":"ContainerStarted","Data":"5b8379d46db9b6237fc3c7b72e3dd9140c8a60464c846975f6ab91fdd6c18cc1"}
Jul 09 15:38:44.094089 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:44.093931    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f" event={"ID":"66a67b53-128c-4670-b31a-08a0a7e997e4","Type":"ContainerStarted","Data":"2825f2e0edeb48c81e9c161da2c75b037d0f0736ec9853d7d43e7a1d525045e2"}
Jul 09 15:38:44.094089 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:44.093977    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:38:44.116563 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:44.116512    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f" podStartSLOduration=1.572207541 podStartE2EDuration="2.116495565s" totalImagesPullingTime="544.288024ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:38:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:38:44.113959958 +0000 UTC m=+2174.447389926" watchObservedRunningTime="2026-07-09 15:38:44.116495565 +0000 UTC m=+2174.449925556"
Jul 09 15:38:50.105774 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:50.105739    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:38:50.489092 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:50.489063    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"]
Jul 09 15:38:50.492612 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:50.492589    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:38:50.496285 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:50.496265    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a65149-serving-cert\""
Jul 09 15:38:50.511472 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:50.511445    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"]
Jul 09 15:38:50.605364 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:50.605327    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sbdj\" (UniqueName: \"kubernetes.io/projected/0663cb26-4a4f-4632-91c5-91492996de58-kube-api-access-6sbdj\") pod \"maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv\" (UID: \"0663cb26-4a4f-4632-91c5-91492996de58\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:38:50.605558 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:50.605376    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0663cb26-4a4f-4632-91c5-91492996de58-openshift-service-ca\") pod \"maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv\" (UID: \"0663cb26-4a4f-4632-91c5-91492996de58\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:38:50.605558 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:50.605410    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0663cb26-4a4f-4632-91c5-91492996de58-maas-api-tls\") pod \"maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv\" (UID: \"0663cb26-4a4f-4632-91c5-91492996de58\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:38:50.706633 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:50.706601    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0663cb26-4a4f-4632-91c5-91492996de58-maas-api-tls\") pod \"maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv\" (UID: \"0663cb26-4a4f-4632-91c5-91492996de58\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:38:50.706838 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:50.706743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sbdj\" (UniqueName: \"kubernetes.io/projected/0663cb26-4a4f-4632-91c5-91492996de58-kube-api-access-6sbdj\") pod \"maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv\" (UID: \"0663cb26-4a4f-4632-91c5-91492996de58\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:38:50.706838 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:38:50.706761    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-serving-cert: secret "maas-api-e2e-api-b-a65149-serving-cert" not found
Jul 09 15:38:50.706838 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:50.706787    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0663cb26-4a4f-4632-91c5-91492996de58-openshift-service-ca\") pod \"maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv\" (UID: \"0663cb26-4a4f-4632-91c5-91492996de58\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:38:50.706838 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:38:50.706823    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0663cb26-4a4f-4632-91c5-91492996de58-maas-api-tls podName:0663cb26-4a4f-4632-91c5-91492996de58 nodeName:}" failed. No retries permitted until 2026-07-09 15:38:51.206801782 +0000 UTC m=+2181.540231726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0663cb26-4a4f-4632-91c5-91492996de58-maas-api-tls") pod "maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv" (UID: "0663cb26-4a4f-4632-91c5-91492996de58") : secret "maas-api-e2e-api-b-a65149-serving-cert" not found
Jul 09 15:38:50.707586 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:50.707551    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0663cb26-4a4f-4632-91c5-91492996de58-openshift-service-ca\") pod \"maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv\" (UID: \"0663cb26-4a4f-4632-91c5-91492996de58\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:38:50.721444 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:50.721409    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sbdj\" (UniqueName: \"kubernetes.io/projected/0663cb26-4a4f-4632-91c5-91492996de58-kube-api-access-6sbdj\") pod \"maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv\" (UID: \"0663cb26-4a4f-4632-91c5-91492996de58\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:38:51.209897 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:51.209858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0663cb26-4a4f-4632-91c5-91492996de58-maas-api-tls\") pod \"maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv\" (UID: \"0663cb26-4a4f-4632-91c5-91492996de58\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:38:51.210383 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:38:51.210030    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-serving-cert: secret "maas-api-e2e-api-b-a65149-serving-cert" not found
Jul 09 15:38:51.210383 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:38:51.210123    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0663cb26-4a4f-4632-91c5-91492996de58-maas-api-tls podName:0663cb26-4a4f-4632-91c5-91492996de58 nodeName:}" failed. No retries permitted until 2026-07-09 15:38:52.210106026 +0000 UTC m=+2182.543535969 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0663cb26-4a4f-4632-91c5-91492996de58-maas-api-tls") pod "maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv" (UID: "0663cb26-4a4f-4632-91c5-91492996de58") : secret "maas-api-e2e-api-b-a65149-serving-cert" not found
Jul 09 15:38:52.206658 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:52.206626    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:38:52.219322 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:52.219287    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0663cb26-4a4f-4632-91c5-91492996de58-maas-api-tls\") pod \"maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv\" (UID: \"0663cb26-4a4f-4632-91c5-91492996de58\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:38:52.222492 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:52.222463    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0663cb26-4a4f-4632-91c5-91492996de58-maas-api-tls\") pod \"maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv\" (UID: \"0663cb26-4a4f-4632-91c5-91492996de58\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:38:52.302704 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:52.302643    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:38:52.476742 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:52.476716    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"]
Jul 09 15:38:52.479022 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:38:52.478976    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0663cb26_4a4f_4632_91c5_91492996de58.slice/crio-f720d89d4895b27147745b8e3204746f7f6b59ed10992961c2305c943c6130f9 WatchSource:0}: Error finding container f720d89d4895b27147745b8e3204746f7f6b59ed10992961c2305c943c6130f9: Status 404 returned error can't find the container with id f720d89d4895b27147745b8e3204746f7f6b59ed10992961c2305c943c6130f9
Jul 09 15:38:53.139434 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:53.139344    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv" event={"ID":"0663cb26-4a4f-4632-91c5-91492996de58","Type":"ContainerStarted","Data":"28e8c04f08680b3bef06175d260029fe57f9bf265145b9c78a6f653036c7f983"}
Jul 09 15:38:53.139434 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:53.139399    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv" event={"ID":"0663cb26-4a4f-4632-91c5-91492996de58","Type":"ContainerStarted","Data":"f720d89d4895b27147745b8e3204746f7f6b59ed10992961c2305c943c6130f9"}
Jul 09 15:38:53.139629 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:53.139471    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:38:53.163380 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:53.163334    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv" podStartSLOduration=2.834943187 podStartE2EDuration="3.163320028s" totalImagesPullingTime="328.376841ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:38:53.161462377 +0000 UTC m=+2183.494892343" watchObservedRunningTime="2026-07-09 15:38:53.163320028 +0000 UTC m=+2183.496749994"
Jul 09 15:38:59.151217 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:38:59.151185    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:39:04.315876 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:04.315843    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"]
Jul 09 15:39:04.318285 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:04.316078    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f" podUID="66a67b53-128c-4670-b31a-08a0a7e997e4" containerName="maas-api" containerID="cri-o://5b8379d46db9b6237fc3c7b72e3dd9140c8a60464c846975f6ab91fdd6c18cc1" gracePeriod=30
Jul 09 15:39:04.562439 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:04.562415    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:39:04.630990 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:04.630914    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66a67b53-128c-4670-b31a-08a0a7e997e4-openshift-service-ca\") pod \"66a67b53-128c-4670-b31a-08a0a7e997e4\" (UID: \"66a67b53-128c-4670-b31a-08a0a7e997e4\") "
Jul 09 15:39:04.630990 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:04.630974    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66a67b53-128c-4670-b31a-08a0a7e997e4-maas-api-tls\") pod \"66a67b53-128c-4670-b31a-08a0a7e997e4\" (UID: \"66a67b53-128c-4670-b31a-08a0a7e997e4\") "
Jul 09 15:39:04.631163 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:04.631019    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcp75\" (UniqueName: \"kubernetes.io/projected/66a67b53-128c-4670-b31a-08a0a7e997e4-kube-api-access-hcp75\") pod \"66a67b53-128c-4670-b31a-08a0a7e997e4\" (UID: \"66a67b53-128c-4670-b31a-08a0a7e997e4\") "
Jul 09 15:39:04.631301 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:04.631276    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66a67b53-128c-4670-b31a-08a0a7e997e4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "66a67b53-128c-4670-b31a-08a0a7e997e4" (UID: "66a67b53-128c-4670-b31a-08a0a7e997e4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:39:04.633074 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:04.633049    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66a67b53-128c-4670-b31a-08a0a7e997e4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "66a67b53-128c-4670-b31a-08a0a7e997e4" (UID: "66a67b53-128c-4670-b31a-08a0a7e997e4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:39:04.633150 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:04.633096    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66a67b53-128c-4670-b31a-08a0a7e997e4-kube-api-access-hcp75" (OuterVolumeSpecName: "kube-api-access-hcp75") pod "66a67b53-128c-4670-b31a-08a0a7e997e4" (UID: "66a67b53-128c-4670-b31a-08a0a7e997e4"). InnerVolumeSpecName "kube-api-access-hcp75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:39:04.731860 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:04.731830    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66a67b53-128c-4670-b31a-08a0a7e997e4-maas-api-tls\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:39:04.731860 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:04.731854    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcp75\" (UniqueName: \"kubernetes.io/projected/66a67b53-128c-4670-b31a-08a0a7e997e4-kube-api-access-hcp75\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:39:04.731860 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:04.731863    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66a67b53-128c-4670-b31a-08a0a7e997e4-openshift-service-ca\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:39:05.198282 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:05.198241    2579 generic.go:356] "Generic (PLEG): container finished" podID="66a67b53-128c-4670-b31a-08a0a7e997e4" containerID="5b8379d46db9b6237fc3c7b72e3dd9140c8a60464c846975f6ab91fdd6c18cc1" exitCode=0
Jul 09 15:39:05.198491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:05.198300    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"
Jul 09 15:39:05.198491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:05.198320    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f" event={"ID":"66a67b53-128c-4670-b31a-08a0a7e997e4","Type":"ContainerDied","Data":"5b8379d46db9b6237fc3c7b72e3dd9140c8a60464c846975f6ab91fdd6c18cc1"}
Jul 09 15:39:05.198491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:05.198378    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f" event={"ID":"66a67b53-128c-4670-b31a-08a0a7e997e4","Type":"ContainerDied","Data":"2825f2e0edeb48c81e9c161da2c75b037d0f0736ec9853d7d43e7a1d525045e2"}
Jul 09 15:39:05.198491 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:05.198402    2579 scope.go:117] "RemoveContainer" containerID="5b8379d46db9b6237fc3c7b72e3dd9140c8a60464c846975f6ab91fdd6c18cc1"
Jul 09 15:39:05.208376 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:05.208355    2579 scope.go:117] "RemoveContainer" containerID="5b8379d46db9b6237fc3c7b72e3dd9140c8a60464c846975f6ab91fdd6c18cc1"
Jul 09 15:39:05.208694 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:39:05.208664    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b8379d46db9b6237fc3c7b72e3dd9140c8a60464c846975f6ab91fdd6c18cc1\": container with ID starting with 5b8379d46db9b6237fc3c7b72e3dd9140c8a60464c846975f6ab91fdd6c18cc1 not found: ID does not exist" containerID="5b8379d46db9b6237fc3c7b72e3dd9140c8a60464c846975f6ab91fdd6c18cc1"
Jul 09 15:39:05.208760 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:05.208711    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b8379d46db9b6237fc3c7b72e3dd9140c8a60464c846975f6ab91fdd6c18cc1"} err="failed to get container status \"5b8379d46db9b6237fc3c7b72e3dd9140c8a60464c846975f6ab91fdd6c18cc1\": rpc error: code = NotFound desc = could not find container \"5b8379d46db9b6237fc3c7b72e3dd9140c8a60464c846975f6ab91fdd6c18cc1\": container with ID starting with 5b8379d46db9b6237fc3c7b72e3dd9140c8a60464c846975f6ab91fdd6c18cc1 not found: ID does not exist"
Jul 09 15:39:05.221672 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:05.221643    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"]
Jul 09 15:39:05.226461 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:05.226438    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0164f1-5bfbdd6888-mbh7f"]
Jul 09 15:39:06.230394 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:06.230363    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66a67b53-128c-4670-b31a-08a0a7e997e4" path="/var/lib/kubelet/pods/66a67b53-128c-4670-b31a-08a0a7e997e4/volumes"
Jul 09 15:39:08.663062 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:08.663025    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:39:12.488250 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:12.488222    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"]
Jul 09 15:39:12.488635 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:12.488446    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv" podUID="0663cb26-4a4f-4632-91c5-91492996de58" containerName="maas-api" containerID="cri-o://28e8c04f08680b3bef06175d260029fe57f9bf265145b9c78a6f653036c7f983" gracePeriod=30
Jul 09 15:39:12.735428 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:12.735406    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:39:12.798595 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:12.798538    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0663cb26-4a4f-4632-91c5-91492996de58-maas-api-tls\") pod \"0663cb26-4a4f-4632-91c5-91492996de58\" (UID: \"0663cb26-4a4f-4632-91c5-91492996de58\") "
Jul 09 15:39:12.798733 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:12.798594    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0663cb26-4a4f-4632-91c5-91492996de58-openshift-service-ca\") pod \"0663cb26-4a4f-4632-91c5-91492996de58\" (UID: \"0663cb26-4a4f-4632-91c5-91492996de58\") "
Jul 09 15:39:12.798733 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:12.798643    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sbdj\" (UniqueName: \"kubernetes.io/projected/0663cb26-4a4f-4632-91c5-91492996de58-kube-api-access-6sbdj\") pod \"0663cb26-4a4f-4632-91c5-91492996de58\" (UID: \"0663cb26-4a4f-4632-91c5-91492996de58\") "
Jul 09 15:39:12.799014 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:12.798987    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0663cb26-4a4f-4632-91c5-91492996de58-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0663cb26-4a4f-4632-91c5-91492996de58" (UID: "0663cb26-4a4f-4632-91c5-91492996de58"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:39:12.800515 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:12.800496    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0663cb26-4a4f-4632-91c5-91492996de58-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0663cb26-4a4f-4632-91c5-91492996de58" (UID: "0663cb26-4a4f-4632-91c5-91492996de58"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:39:12.800613 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:12.800593    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0663cb26-4a4f-4632-91c5-91492996de58-kube-api-access-6sbdj" (OuterVolumeSpecName: "kube-api-access-6sbdj") pod "0663cb26-4a4f-4632-91c5-91492996de58" (UID: "0663cb26-4a4f-4632-91c5-91492996de58"). InnerVolumeSpecName "kube-api-access-6sbdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:39:12.899502 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:12.899479    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0663cb26-4a4f-4632-91c5-91492996de58-maas-api-tls\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:39:12.899502 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:12.899499    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0663cb26-4a4f-4632-91c5-91492996de58-openshift-service-ca\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:39:12.899623 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:12.899509    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sbdj\" (UniqueName: \"kubernetes.io/projected/0663cb26-4a4f-4632-91c5-91492996de58-kube-api-access-6sbdj\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:39:13.242797 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:13.242760    2579 generic.go:356] "Generic (PLEG): container finished" podID="0663cb26-4a4f-4632-91c5-91492996de58" containerID="28e8c04f08680b3bef06175d260029fe57f9bf265145b9c78a6f653036c7f983" exitCode=0
Jul 09 15:39:13.242957 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:13.242805    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv" event={"ID":"0663cb26-4a4f-4632-91c5-91492996de58","Type":"ContainerDied","Data":"28e8c04f08680b3bef06175d260029fe57f9bf265145b9c78a6f653036c7f983"}
Jul 09 15:39:13.242957 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:13.242842    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv" event={"ID":"0663cb26-4a4f-4632-91c5-91492996de58","Type":"ContainerDied","Data":"f720d89d4895b27147745b8e3204746f7f6b59ed10992961c2305c943c6130f9"}
Jul 09 15:39:13.242957 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:13.242862    2579 scope.go:117] "RemoveContainer" containerID="28e8c04f08680b3bef06175d260029fe57f9bf265145b9c78a6f653036c7f983"
Jul 09 15:39:13.242957 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:13.242806    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"
Jul 09 15:39:13.252201 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:13.252184    2579 scope.go:117] "RemoveContainer" containerID="28e8c04f08680b3bef06175d260029fe57f9bf265145b9c78a6f653036c7f983"
Jul 09 15:39:13.252480 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:39:13.252461    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28e8c04f08680b3bef06175d260029fe57f9bf265145b9c78a6f653036c7f983\": container with ID starting with 28e8c04f08680b3bef06175d260029fe57f9bf265145b9c78a6f653036c7f983 not found: ID does not exist" containerID="28e8c04f08680b3bef06175d260029fe57f9bf265145b9c78a6f653036c7f983"
Jul 09 15:39:13.252550 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:13.252489    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28e8c04f08680b3bef06175d260029fe57f9bf265145b9c78a6f653036c7f983"} err="failed to get container status \"28e8c04f08680b3bef06175d260029fe57f9bf265145b9c78a6f653036c7f983\": rpc error: code = NotFound desc = could not find container \"28e8c04f08680b3bef06175d260029fe57f9bf265145b9c78a6f653036c7f983\": container with ID starting with 28e8c04f08680b3bef06175d260029fe57f9bf265145b9c78a6f653036c7f983 not found: ID does not exist"
Jul 09 15:39:13.272342 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:13.272313    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"]
Jul 09 15:39:13.279208 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:13.279184    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a65149-7948dcf9f9-qmrjv"]
Jul 09 15:39:14.236627 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:14.232911    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0663cb26-4a4f-4632-91c5-91492996de58" path="/var/lib/kubelet/pods/0663cb26-4a4f-4632-91c5-91492996de58/volumes"
Jul 09 15:39:22.162056 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.162009    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:39:22.594210 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.594179    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"]
Jul 09 15:39:22.594538 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.594524    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0663cb26-4a4f-4632-91c5-91492996de58" containerName="maas-api"
Jul 09 15:39:22.594538 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.594539    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0663cb26-4a4f-4632-91c5-91492996de58" containerName="maas-api"
Jul 09 15:39:22.594640 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.594554    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66a67b53-128c-4670-b31a-08a0a7e997e4" containerName="maas-api"
Jul 09 15:39:22.594640 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.594563    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="66a67b53-128c-4670-b31a-08a0a7e997e4" containerName="maas-api"
Jul 09 15:39:22.594640 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.594628    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="66a67b53-128c-4670-b31a-08a0a7e997e4" containerName="maas-api"
Jul 09 15:39:22.594640 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.594639    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0663cb26-4a4f-4632-91c5-91492996de58" containerName="maas-api"
Jul 09 15:39:22.598523 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.598506    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:39:22.601421 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.601395    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-dcc4b5-serving-cert\""
Jul 09 15:39:22.630255 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.630233    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"]
Jul 09 15:39:22.675477 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.675449    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkhq5\" (UniqueName: \"kubernetes.io/projected/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-kube-api-access-xkhq5\") pod \"maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf\" (UID: \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:39:22.675607 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.675486    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-openshift-service-ca\") pod \"maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf\" (UID: \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:39:22.675607 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.675577    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-maas-api-tls\") pod \"maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf\" (UID: \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:39:22.776367 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.776334    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-maas-api-tls\") pod \"maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf\" (UID: \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:39:22.776508 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.776383    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkhq5\" (UniqueName: \"kubernetes.io/projected/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-kube-api-access-xkhq5\") pod \"maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf\" (UID: \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:39:22.776508 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:39:22.776480    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-serving-cert: secret "maas-api-e2e-shared-a-dcc4b5-serving-cert" not found
Jul 09 15:39:22.776508 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.776495    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-openshift-service-ca\") pod \"maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf\" (UID: \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:39:22.776632 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:39:22.776551    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-maas-api-tls podName:9baad3be-2e55-4f50-96bd-d0fdcae39fcf nodeName:}" failed. No retries permitted until 2026-07-09 15:39:23.276533988 +0000 UTC m=+2213.609963930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-maas-api-tls") pod "maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf" (UID: "9baad3be-2e55-4f50-96bd-d0fdcae39fcf") : secret "maas-api-e2e-shared-a-dcc4b5-serving-cert" not found
Jul 09 15:39:22.777158 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.777140    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-openshift-service-ca\") pod \"maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf\" (UID: \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:39:22.794985 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:22.794963    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkhq5\" (UniqueName: \"kubernetes.io/projected/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-kube-api-access-xkhq5\") pod \"maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf\" (UID: \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:39:23.281943 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:23.281905    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-maas-api-tls\") pod \"maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf\" (UID: \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:39:23.284277 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:23.284257    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-maas-api-tls\") pod \"maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf\" (UID: \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:39:23.508869 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:23.508832    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:39:23.654611 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:23.654581    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"]
Jul 09 15:39:23.654767 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:39:23.654715    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9baad3be_2e55_4f50_96bd_d0fdcae39fcf.slice/crio-35597123fdfe027e0e0a468ab62eb297afce1fd209b508c06db76e1a86be967d WatchSource:0}: Error finding container 35597123fdfe027e0e0a468ab62eb297afce1fd209b508c06db76e1a86be967d: Status 404 returned error can't find the container with id 35597123fdfe027e0e0a468ab62eb297afce1fd209b508c06db76e1a86be967d
Jul 09 15:39:24.295085 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:24.295051    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf" event={"ID":"9baad3be-2e55-4f50-96bd-d0fdcae39fcf","Type":"ContainerStarted","Data":"35597123fdfe027e0e0a468ab62eb297afce1fd209b508c06db76e1a86be967d"}
Jul 09 15:39:26.305379 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:26.305297    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf" event={"ID":"9baad3be-2e55-4f50-96bd-d0fdcae39fcf","Type":"ContainerStarted","Data":"9f4988e44f30220b8a08b5065cc91778523b6dcc2b3c2c399ba25cf455028cc7"}
Jul 09 15:39:26.305755 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:26.305433    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:39:26.329507 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:26.329462    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf" podStartSLOduration=2.035263482 podStartE2EDuration="4.329448424s" totalImagesPullingTime="2.294184942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:39:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:39:26.326939768 +0000 UTC m=+2216.660369734" watchObservedRunningTime="2026-07-09 15:39:26.329448424 +0000 UTC m=+2216.662878402"
Jul 09 15:39:30.630122 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:30.630046    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"]
Jul 09 15:39:30.635434 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:30.635406    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"
Jul 09 15:39:30.639453 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:30.639435    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-863e67-serving-cert\""
Jul 09 15:39:30.645022 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:30.644993    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"]
Jul 09 15:39:30.745832 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:30.745803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zllx5\" (UniqueName: \"kubernetes.io/projected/2f5f5738-3405-4869-81ec-c258f47531ba-kube-api-access-zllx5\") pod \"maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45\" (UID: \"2f5f5738-3405-4869-81ec-c258f47531ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"
Jul 09 15:39:30.746007 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:30.745856    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f5f5738-3405-4869-81ec-c258f47531ba-maas-api-tls\") pod \"maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45\" (UID: \"2f5f5738-3405-4869-81ec-c258f47531ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"
Jul 09 15:39:30.746007 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:30.745880    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f5f5738-3405-4869-81ec-c258f47531ba-openshift-service-ca\") pod \"maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45\" (UID: \"2f5f5738-3405-4869-81ec-c258f47531ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"
Jul 09 15:39:30.847130 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:30.847099    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zllx5\" (UniqueName: \"kubernetes.io/projected/2f5f5738-3405-4869-81ec-c258f47531ba-kube-api-access-zllx5\") pod \"maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45\" (UID: \"2f5f5738-3405-4869-81ec-c258f47531ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"
Jul 09 15:39:30.847308 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:30.847139    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f5f5738-3405-4869-81ec-c258f47531ba-maas-api-tls\") pod \"maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45\" (UID: \"2f5f5738-3405-4869-81ec-c258f47531ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"
Jul 09 15:39:30.847308 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:30.847158    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f5f5738-3405-4869-81ec-c258f47531ba-openshift-service-ca\") pod \"maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45\" (UID: \"2f5f5738-3405-4869-81ec-c258f47531ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"
Jul 09 15:39:30.847923 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:30.847901    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f5f5738-3405-4869-81ec-c258f47531ba-openshift-service-ca\") pod \"maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45\" (UID: \"2f5f5738-3405-4869-81ec-c258f47531ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"
Jul 09 15:39:30.849638 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:30.849611    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f5f5738-3405-4869-81ec-c258f47531ba-maas-api-tls\") pod \"maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45\" (UID: \"2f5f5738-3405-4869-81ec-c258f47531ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"
Jul 09 15:39:30.861923 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:30.861897    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zllx5\" (UniqueName: \"kubernetes.io/projected/2f5f5738-3405-4869-81ec-c258f47531ba-kube-api-access-zllx5\") pod \"maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45\" (UID: \"2f5f5738-3405-4869-81ec-c258f47531ba\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"
Jul 09 15:39:30.948466 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:30.948438    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"
Jul 09 15:39:31.089719 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:31.089672    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"]
Jul 09 15:39:31.093156 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:39:31.093122    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f5f5738_3405_4869_81ec_c258f47531ba.slice/crio-6b40886ff3bd2709aa3251e3fb3f50c0df9e0d39aafe1ddc72623ff2db4ba9e1 WatchSource:0}: Error finding container 6b40886ff3bd2709aa3251e3fb3f50c0df9e0d39aafe1ddc72623ff2db4ba9e1: Status 404 returned error can't find the container with id 6b40886ff3bd2709aa3251e3fb3f50c0df9e0d39aafe1ddc72623ff2db4ba9e1
Jul 09 15:39:31.330486 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:31.330399    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45" event={"ID":"2f5f5738-3405-4869-81ec-c258f47531ba","Type":"ContainerStarted","Data":"6b40886ff3bd2709aa3251e3fb3f50c0df9e0d39aafe1ddc72623ff2db4ba9e1"}
Jul 09 15:39:32.316380 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:32.316351    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:39:32.336534 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:32.336504    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45" event={"ID":"2f5f5738-3405-4869-81ec-c258f47531ba","Type":"ContainerStarted","Data":"b5281957ca3f6e0eefbe01284bd00f48d74aa45854df45c0189101d33cfdb7d4"}
Jul 09 15:39:32.336666 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:32.336633    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"
Jul 09 15:39:32.384773 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:32.384731    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45" podStartSLOduration=2.013182947 podStartE2EDuration="2.384716391s" totalImagesPullingTime="371.533444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:39:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:39:32.383354597 +0000 UTC m=+2222.716784563" watchObservedRunningTime="2026-07-09 15:39:32.384716391 +0000 UTC m=+2222.718146353"
Jul 09 15:39:36.079098 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:36.079069    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:39:38.346663 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:39:38.346636    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"
Jul 09 15:42:30.274619 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:30.274583    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:42:30.279287 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:30.279258    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n8ctb_1c59bea2-03b5-4242-8f9a-9510da7d4eca/ovn-acl-logging/0.log"
Jul 09 15:42:41.721800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:41.721764    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"]
Jul 09 15:42:41.722230 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:41.721954    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf" podUID="9baad3be-2e55-4f50-96bd-d0fdcae39fcf" containerName="maas-api" containerID="cri-o://9f4988e44f30220b8a08b5065cc91778523b6dcc2b3c2c399ba25cf455028cc7" gracePeriod=30
Jul 09 15:42:41.961900 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:41.961872    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:42:42.030472 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.030386    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-maas-api-tls\") pod \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\" (UID: \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\") "
Jul 09 15:42:42.030472 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.030449    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkhq5\" (UniqueName: \"kubernetes.io/projected/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-kube-api-access-xkhq5\") pod \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\" (UID: \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\") "
Jul 09 15:42:42.030721 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.030493    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-openshift-service-ca\") pod \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\" (UID: \"9baad3be-2e55-4f50-96bd-d0fdcae39fcf\") "
Jul 09 15:42:42.030911 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.030884    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9baad3be-2e55-4f50-96bd-d0fdcae39fcf" (UID: "9baad3be-2e55-4f50-96bd-d0fdcae39fcf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:42:42.032588 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.032558    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9baad3be-2e55-4f50-96bd-d0fdcae39fcf" (UID: "9baad3be-2e55-4f50-96bd-d0fdcae39fcf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:42:42.032752 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.032637    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-kube-api-access-xkhq5" (OuterVolumeSpecName: "kube-api-access-xkhq5") pod "9baad3be-2e55-4f50-96bd-d0fdcae39fcf" (UID: "9baad3be-2e55-4f50-96bd-d0fdcae39fcf"). InnerVolumeSpecName "kube-api-access-xkhq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:42:42.131745 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.131709    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-maas-api-tls\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:42:42.131745 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.131740    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkhq5\" (UniqueName: \"kubernetes.io/projected/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-kube-api-access-xkhq5\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:42:42.131745 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.131750    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9baad3be-2e55-4f50-96bd-d0fdcae39fcf-openshift-service-ca\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:42:42.226676 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.226643    2579 generic.go:356] "Generic (PLEG): container finished" podID="9baad3be-2e55-4f50-96bd-d0fdcae39fcf" containerID="9f4988e44f30220b8a08b5065cc91778523b6dcc2b3c2c399ba25cf455028cc7" exitCode=0
Jul 09 15:42:42.226837 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.226774    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"
Jul 09 15:42:42.231494 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.231457    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf" event={"ID":"9baad3be-2e55-4f50-96bd-d0fdcae39fcf","Type":"ContainerDied","Data":"9f4988e44f30220b8a08b5065cc91778523b6dcc2b3c2c399ba25cf455028cc7"}
Jul 09 15:42:42.231639 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.231512    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf" event={"ID":"9baad3be-2e55-4f50-96bd-d0fdcae39fcf","Type":"ContainerDied","Data":"35597123fdfe027e0e0a468ab62eb297afce1fd209b508c06db76e1a86be967d"}
Jul 09 15:42:42.231639 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.231543    2579 scope.go:117] "RemoveContainer" containerID="9f4988e44f30220b8a08b5065cc91778523b6dcc2b3c2c399ba25cf455028cc7"
Jul 09 15:42:42.241446 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.241429    2579 scope.go:117] "RemoveContainer" containerID="9f4988e44f30220b8a08b5065cc91778523b6dcc2b3c2c399ba25cf455028cc7"
Jul 09 15:42:42.241713 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:42:42.241678    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f4988e44f30220b8a08b5065cc91778523b6dcc2b3c2c399ba25cf455028cc7\": container with ID starting with 9f4988e44f30220b8a08b5065cc91778523b6dcc2b3c2c399ba25cf455028cc7 not found: ID does not exist" containerID="9f4988e44f30220b8a08b5065cc91778523b6dcc2b3c2c399ba25cf455028cc7"
Jul 09 15:42:42.241816 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.241722    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f4988e44f30220b8a08b5065cc91778523b6dcc2b3c2c399ba25cf455028cc7"} err="failed to get container status \"9f4988e44f30220b8a08b5065cc91778523b6dcc2b3c2c399ba25cf455028cc7\": rpc error: code = NotFound desc = could not find container \"9f4988e44f30220b8a08b5065cc91778523b6dcc2b3c2c399ba25cf455028cc7\": container with ID starting with 9f4988e44f30220b8a08b5065cc91778523b6dcc2b3c2c399ba25cf455028cc7 not found: ID does not exist"
Jul 09 15:42:42.268150 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.268125    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"]
Jul 09 15:42:42.271856 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:42.271834    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-dcc4b5-55ffd8754b-xtlzf"]
Jul 09 15:42:44.230850 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:44.230820    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9baad3be-2e55-4f50-96bd-d0fdcae39fcf" path="/var/lib/kubelet/pods/9baad3be-2e55-4f50-96bd-d0fdcae39fcf/volumes"
Jul 09 15:42:45.182364 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:45.182329    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:42:50.275314 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:50.275283    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"]
Jul 09 15:42:50.275833 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:50.275468    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45" podUID="2f5f5738-3405-4869-81ec-c258f47531ba" containerName="maas-api" containerID="cri-o://b5281957ca3f6e0eefbe01284bd00f48d74aa45854df45c0189101d33cfdb7d4" gracePeriod=30
Jul 09 15:42:50.525980 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:50.525929    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"
Jul 09 15:42:50.602432 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:50.602401    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f5f5738-3405-4869-81ec-c258f47531ba-openshift-service-ca\") pod \"2f5f5738-3405-4869-81ec-c258f47531ba\" (UID: \"2f5f5738-3405-4869-81ec-c258f47531ba\") "
Jul 09 15:42:50.602595 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:50.602466    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f5f5738-3405-4869-81ec-c258f47531ba-maas-api-tls\") pod \"2f5f5738-3405-4869-81ec-c258f47531ba\" (UID: \"2f5f5738-3405-4869-81ec-c258f47531ba\") "
Jul 09 15:42:50.602595 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:50.602490    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zllx5\" (UniqueName: \"kubernetes.io/projected/2f5f5738-3405-4869-81ec-c258f47531ba-kube-api-access-zllx5\") pod \"2f5f5738-3405-4869-81ec-c258f47531ba\" (UID: \"2f5f5738-3405-4869-81ec-c258f47531ba\") "
Jul 09 15:42:50.602800 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:50.602777    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f5f5738-3405-4869-81ec-c258f47531ba-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f5f5738-3405-4869-81ec-c258f47531ba" (UID: "2f5f5738-3405-4869-81ec-c258f47531ba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:42:50.602927 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:50.602915    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f5f5738-3405-4869-81ec-c258f47531ba-openshift-service-ca\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:42:50.604547 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:50.604526    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f5f5738-3405-4869-81ec-c258f47531ba-kube-api-access-zllx5" (OuterVolumeSpecName: "kube-api-access-zllx5") pod "2f5f5738-3405-4869-81ec-c258f47531ba" (UID: "2f5f5738-3405-4869-81ec-c258f47531ba"). InnerVolumeSpecName "kube-api-access-zllx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:42:50.604547 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:50.604536    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f5f5738-3405-4869-81ec-c258f47531ba-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f5f5738-3405-4869-81ec-c258f47531ba" (UID: "2f5f5738-3405-4869-81ec-c258f47531ba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:42:50.703473 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:50.703444    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f5f5738-3405-4869-81ec-c258f47531ba-maas-api-tls\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:42:50.703473 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:50.703471    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zllx5\" (UniqueName: \"kubernetes.io/projected/2f5f5738-3405-4869-81ec-c258f47531ba-kube-api-access-zllx5\") on node \"ip-10-0-129-152.ec2.internal\" DevicePath \"\""
Jul 09 15:42:51.271921 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:51.271890    2579 generic.go:356] "Generic (PLEG): container finished" podID="2f5f5738-3405-4869-81ec-c258f47531ba" containerID="b5281957ca3f6e0eefbe01284bd00f48d74aa45854df45c0189101d33cfdb7d4" exitCode=0
Jul 09 15:42:51.272076 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:51.271932    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"
Jul 09 15:42:51.272076 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:51.271959    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45" event={"ID":"2f5f5738-3405-4869-81ec-c258f47531ba","Type":"ContainerDied","Data":"b5281957ca3f6e0eefbe01284bd00f48d74aa45854df45c0189101d33cfdb7d4"}
Jul 09 15:42:51.272076 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:51.272001    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45" event={"ID":"2f5f5738-3405-4869-81ec-c258f47531ba","Type":"ContainerDied","Data":"6b40886ff3bd2709aa3251e3fb3f50c0df9e0d39aafe1ddc72623ff2db4ba9e1"}
Jul 09 15:42:51.272076 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:51.272022    2579 scope.go:117] "RemoveContainer" containerID="b5281957ca3f6e0eefbe01284bd00f48d74aa45854df45c0189101d33cfdb7d4"
Jul 09 15:42:51.283340 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:51.283148    2579 scope.go:117] "RemoveContainer" containerID="b5281957ca3f6e0eefbe01284bd00f48d74aa45854df45c0189101d33cfdb7d4"
Jul 09 15:42:51.283552 ip-10-0-129-152 kubenswrapper[2579]: E0709 15:42:51.283423    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5281957ca3f6e0eefbe01284bd00f48d74aa45854df45c0189101d33cfdb7d4\": container with ID starting with b5281957ca3f6e0eefbe01284bd00f48d74aa45854df45c0189101d33cfdb7d4 not found: ID does not exist" containerID="b5281957ca3f6e0eefbe01284bd00f48d74aa45854df45c0189101d33cfdb7d4"
Jul 09 15:42:51.283552 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:51.283462    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5281957ca3f6e0eefbe01284bd00f48d74aa45854df45c0189101d33cfdb7d4"} err="failed to get container status \"b5281957ca3f6e0eefbe01284bd00f48d74aa45854df45c0189101d33cfdb7d4\": rpc error: code = NotFound desc = could not find container \"b5281957ca3f6e0eefbe01284bd00f48d74aa45854df45c0189101d33cfdb7d4\": container with ID starting with b5281957ca3f6e0eefbe01284bd00f48d74aa45854df45c0189101d33cfdb7d4 not found: ID does not exist"
Jul 09 15:42:51.298298 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:51.298273    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"]
Jul 09 15:42:51.301401 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:51.301383    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-863e67-6fc8d5c65c-k8j45"]
Jul 09 15:42:52.231153 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:52.231075    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f5f5738-3405-4869-81ec-c258f47531ba" path="/var/lib/kubelet/pods/2f5f5738-3405-4869-81ec-c258f47531ba/volumes"
Jul 09 15:42:53.926739 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:53.926700    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-74r2z"]
Jul 09 15:42:57.598929 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:42:57.598898    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-79794498cf-snk7n_f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc/authorino/0.log"
Jul 09 15:43:02.962539 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:02.962500    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jdx44_1c395aa0-45df-428a-aace-f93e3325e9f8/manager/0.log"
Jul 09 15:43:03.401846 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:03.401749    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qdln4_5407f877-73d9-49b5-856b-a880d09f3963/manager/0.log"
Jul 09 15:43:03.661036 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:03.660961    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66b8fbd475-8gj8v_f2917027-3953-4933-b7a4-7b5f7c45d6ef/copy-manifests/0.log"
Jul 09 15:43:03.669486 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:03.669465    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66b8fbd475-8gj8v_f2917027-3953-4933-b7a4-7b5f7c45d6ef/manager/0.log"
Jul 09 15:43:04.659641 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:04.659610    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b_2af2c981-2b16-4b6b-9340-e7185e696eb9/extract/0.log"
Jul 09 15:43:04.668546 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:04.668523    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b_2af2c981-2b16-4b6b-9340-e7185e696eb9/util/0.log"
Jul 09 15:43:04.675198 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:04.675169    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b_2af2c981-2b16-4b6b-9340-e7185e696eb9/pull/0.log"
Jul 09 15:43:04.810396 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:04.810363    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7_9f891183-ad02-4bcf-85a7-18523561eb65/util/0.log"
Jul 09 15:43:04.822074 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:04.822041    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7_9f891183-ad02-4bcf-85a7-18523561eb65/pull/0.log"
Jul 09 15:43:04.836438 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:04.836412    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7_9f891183-ad02-4bcf-85a7-18523561eb65/extract/0.log"
Jul 09 15:43:04.969408 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:04.969378    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg_d503cc81-1d1c-451f-ba63-e369839ba04e/util/0.log"
Jul 09 15:43:04.987360 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:04.987328    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg_d503cc81-1d1c-451f-ba63-e369839ba04e/pull/0.log"
Jul 09 15:43:05.004057 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:05.004032    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg_d503cc81-1d1c-451f-ba63-e369839ba04e/extract/0.log"
Jul 09 15:43:05.129227 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:05.129189    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb_d5fe34af-cfa8-4342-a611-4e4a1bb9c21e/util/0.log"
Jul 09 15:43:05.137798 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:05.137774    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb_d5fe34af-cfa8-4342-a611-4e4a1bb9c21e/pull/0.log"
Jul 09 15:43:05.149497 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:05.149478    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb_d5fe34af-cfa8-4342-a611-4e4a1bb9c21e/extract/0.log"
Jul 09 15:43:05.292222 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:05.292137    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-79794498cf-snk7n_f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc/authorino/0.log"
Jul 09 15:43:05.421414 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:05.421379    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-gf42b_f8c4742e-2296-4bf5-a111-a2b828175a6f/manager/0.log"
Jul 09 15:43:05.553191 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:05.553096    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-x296k_ee481a72-01ae-45ac-bdc9-9950abe4421f/manager/0.log"
Jul 09 15:43:05.676761 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:05.676729    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gwz7t_646260f4-cd3e-4a8e-855b-71d52fe2b470/kuadrant-console-plugin/0.log"
Jul 09 15:43:05.824896 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:05.824818    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-8kqc4_82250586-da6f-4184-8741-2f6ae40f638c/registry-server/0.log"
Jul 09 15:43:06.111844 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:06.111757    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-74r2z_563d29fd-00f4-49b4-9bbb-b6ef57f88d41/limitador/0.log"
Jul 09 15:43:06.243680 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:06.243649    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-hnx2c_ae42b068-625d-4729-bf26-8468ea371a8f/manager/0.log"
Jul 09 15:43:06.928503 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:06.928470    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b8bb9d8f7-s5tjx_8f8da16e-38d9-48b8-87f0-4925aa46ed7d/kube-auth-proxy/0.log"
Jul 09 15:43:07.341672 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:07.341570    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qhk65_faa2e895-99fe-471d-a028-17843034d623/payload-pre-processing/0.log"
Jul 09 15:43:08.116472 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:08.116437    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-s7pwb_3437421b-7a2d-4ddb-ba57-d3cff910c40c/main/0.log"
Jul 09 15:43:08.377553 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:08.377472    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db2gm4j_cf2d6765-3d89-4a85-9770-1ed221fde97a/main/0.log"
Jul 09 15:43:08.663652 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:08.663565    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-bq6pk_13834160-216c-420e-ad87-5b9a1a780b03/main/0.log"
Jul 09 15:43:13.052288 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.052246    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v4z7t/must-gather-d986l"]
Jul 09 15:43:13.052798 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.052781    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f5f5738-3405-4869-81ec-c258f47531ba" containerName="maas-api"
Jul 09 15:43:13.052879 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.052801    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f5f5738-3405-4869-81ec-c258f47531ba" containerName="maas-api"
Jul 09 15:43:13.052879 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.052822    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9baad3be-2e55-4f50-96bd-d0fdcae39fcf" containerName="maas-api"
Jul 09 15:43:13.052879 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.052831    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9baad3be-2e55-4f50-96bd-d0fdcae39fcf" containerName="maas-api"
Jul 09 15:43:13.053039 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.052915    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9baad3be-2e55-4f50-96bd-d0fdcae39fcf" containerName="maas-api"
Jul 09 15:43:13.053039 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.052930    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f5f5738-3405-4869-81ec-c258f47531ba" containerName="maas-api"
Jul 09 15:43:13.055951 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.055929    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v4z7t/must-gather-d986l"
Jul 09 15:43:13.066787 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.066763    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v4z7t\"/\"openshift-service-ca.crt\""
Jul 09 15:43:13.066897 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.066820    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-v4z7t\"/\"default-dockercfg-8kg4f\""
Jul 09 15:43:13.067756 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.067743    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v4z7t\"/\"kube-root-ca.crt\""
Jul 09 15:43:13.081401 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.081377    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v4z7t/must-gather-d986l"]
Jul 09 15:43:13.196730 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.196659    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b7706294-e8cb-4bd5-a842-d004fd7d07c4-must-gather-output\") pod \"must-gather-d986l\" (UID: \"b7706294-e8cb-4bd5-a842-d004fd7d07c4\") " pod="openshift-must-gather-v4z7t/must-gather-d986l"
Jul 09 15:43:13.196991 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.196739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5slgq\" (UniqueName: \"kubernetes.io/projected/b7706294-e8cb-4bd5-a842-d004fd7d07c4-kube-api-access-5slgq\") pod \"must-gather-d986l\" (UID: \"b7706294-e8cb-4bd5-a842-d004fd7d07c4\") " pod="openshift-must-gather-v4z7t/must-gather-d986l"
Jul 09 15:43:13.297980 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.297936    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b7706294-e8cb-4bd5-a842-d004fd7d07c4-must-gather-output\") pod \"must-gather-d986l\" (UID: \"b7706294-e8cb-4bd5-a842-d004fd7d07c4\") " pod="openshift-must-gather-v4z7t/must-gather-d986l"
Jul 09 15:43:13.297980 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.297981    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5slgq\" (UniqueName: \"kubernetes.io/projected/b7706294-e8cb-4bd5-a842-d004fd7d07c4-kube-api-access-5slgq\") pod \"must-gather-d986l\" (UID: \"b7706294-e8cb-4bd5-a842-d004fd7d07c4\") " pod="openshift-must-gather-v4z7t/must-gather-d986l"
Jul 09 15:43:13.298378 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.298354    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b7706294-e8cb-4bd5-a842-d004fd7d07c4-must-gather-output\") pod \"must-gather-d986l\" (UID: \"b7706294-e8cb-4bd5-a842-d004fd7d07c4\") " pod="openshift-must-gather-v4z7t/must-gather-d986l"
Jul 09 15:43:13.310895 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.310833    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5slgq\" (UniqueName: \"kubernetes.io/projected/b7706294-e8cb-4bd5-a842-d004fd7d07c4-kube-api-access-5slgq\") pod \"must-gather-d986l\" (UID: \"b7706294-e8cb-4bd5-a842-d004fd7d07c4\") " pod="openshift-must-gather-v4z7t/must-gather-d986l"
Jul 09 15:43:13.365271 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.365235    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v4z7t/must-gather-d986l"
Jul 09 15:43:13.514900 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.514866    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v4z7t/must-gather-d986l"]
Jul 09 15:43:13.516437 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:43:13.516410    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7706294_e8cb_4bd5_a842_d004fd7d07c4.slice/crio-5ea1ebbeece9000116ebd3755b8ed423de0916820aab05e7e1b20e3ea576194a WatchSource:0}: Error finding container 5ea1ebbeece9000116ebd3755b8ed423de0916820aab05e7e1b20e3ea576194a: Status 404 returned error can't find the container with id 5ea1ebbeece9000116ebd3755b8ed423de0916820aab05e7e1b20e3ea576194a
Jul 09 15:43:13.518574 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:13.518558    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 15:43:14.386650 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:14.386616    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v4z7t/must-gather-d986l" event={"ID":"b7706294-e8cb-4bd5-a842-d004fd7d07c4","Type":"ContainerStarted","Data":"5ea1ebbeece9000116ebd3755b8ed423de0916820aab05e7e1b20e3ea576194a"}
Jul 09 15:43:16.398628 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:16.398586    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v4z7t/must-gather-d986l" event={"ID":"b7706294-e8cb-4bd5-a842-d004fd7d07c4","Type":"ContainerStarted","Data":"2606e5227a1193f2d6e45cecc26e778badb00789344c5415747ed235e8f25694"}
Jul 09 15:43:16.398628 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:16.398630    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v4z7t/must-gather-d986l" event={"ID":"b7706294-e8cb-4bd5-a842-d004fd7d07c4","Type":"ContainerStarted","Data":"921a8b49474446b1ceca10d310b6738eee8ebf2e1c2c8e3b39a364491af7d368"}
Jul 09 15:43:16.420457 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:16.420391    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-v4z7t/must-gather-d986l" podStartSLOduration=1.127488394 podStartE2EDuration="3.420369725s" totalImagesPullingTime="2.292881331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:43:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:43:16.417853689 +0000 UTC m=+2446.751283658" watchObservedRunningTime="2026-07-09 15:43:16.420369725 +0000 UTC m=+2446.753799690"
Jul 09 15:43:17.840076 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:17.840043    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6ltzh_a4e083b0-d118-451b-8655-30df5353d5ed/global-pull-secret-syncer/0.log"
Jul 09 15:43:18.444188 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:18.444155    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-r6r56_7feb5924-8488-453b-8093-63183e5c9462/konnectivity-agent/0.log"
Jul 09 15:43:18.572424 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:18.572390    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-152.ec2.internal_e37ba62c1a936e070da56c82d756fe21/haproxy/0.log"
Jul 09 15:43:22.287469 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:22.287440    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b_2af2c981-2b16-4b6b-9340-e7185e696eb9/extract/0.log"
Jul 09 15:43:22.332044 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:22.332011    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b_2af2c981-2b16-4b6b-9340-e7185e696eb9/util/0.log"
Jul 09 15:43:22.371805 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:22.371777    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759x996b_2af2c981-2b16-4b6b-9340-e7185e696eb9/pull/0.log"
Jul 09 15:43:22.415090 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:22.415063    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7_9f891183-ad02-4bcf-85a7-18523561eb65/extract/0.log"
Jul 09 15:43:22.454236 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:22.454205    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7_9f891183-ad02-4bcf-85a7-18523561eb65/util/0.log"
Jul 09 15:43:22.495325 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:22.495287    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vsqm7_9f891183-ad02-4bcf-85a7-18523561eb65/pull/0.log"
Jul 09 15:43:22.561740 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:22.561634    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg_d503cc81-1d1c-451f-ba63-e369839ba04e/extract/0.log"
Jul 09 15:43:22.595096 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:22.595069    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg_d503cc81-1d1c-451f-ba63-e369839ba04e/util/0.log"
Jul 09 15:43:22.629212 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:22.629187    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hvtdg_d503cc81-1d1c-451f-ba63-e369839ba04e/pull/0.log"
Jul 09 15:43:22.684121 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:22.684094    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb_d5fe34af-cfa8-4342-a611-4e4a1bb9c21e/extract/0.log"
Jul 09 15:43:22.742815 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:22.742780    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb_d5fe34af-cfa8-4342-a611-4e4a1bb9c21e/util/0.log"
Jul 09 15:43:22.809240 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:22.809202    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1pj9cb_d5fe34af-cfa8-4342-a611-4e4a1bb9c21e/pull/0.log"
Jul 09 15:43:23.368806 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:23.368772    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-79794498cf-snk7n_f77b8e53-16a8-4c87-b8c1-6ec750cbfdfc/authorino/0.log"
Jul 09 15:43:23.420122 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:23.420084    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-gf42b_f8c4742e-2296-4bf5-a111-a2b828175a6f/manager/0.log"
Jul 09 15:43:23.473976 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:23.473947    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-x296k_ee481a72-01ae-45ac-bdc9-9950abe4421f/manager/0.log"
Jul 09 15:43:23.519405 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:23.519361    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gwz7t_646260f4-cd3e-4a8e-855b-71d52fe2b470/kuadrant-console-plugin/0.log"
Jul 09 15:43:23.588564 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:23.588523    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-8kqc4_82250586-da6f-4184-8741-2f6ae40f638c/registry-server/0.log"
Jul 09 15:43:23.808707 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:23.808666    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-74r2z_563d29fd-00f4-49b4-9bbb-b6ef57f88d41/limitador/0.log"
Jul 09 15:43:23.907296 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:23.907262    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-hnx2c_ae42b068-625d-4729-bf26-8468ea371a8f/manager/0.log"
Jul 09 15:43:26.303126 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:26.303092    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xm86z_aada1234-5724-435a-a621-d686c031135f/node-exporter/0.log"
Jul 09 15:43:26.331258 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:26.331230    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xm86z_aada1234-5724-435a-a621-d686c031135f/kube-rbac-proxy/0.log"
Jul 09 15:43:26.357586 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:26.357483    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xm86z_aada1234-5724-435a-a621-d686c031135f/init-textfile/0.log"
Jul 09 15:43:27.923449 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:27.923414    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"]
Jul 09 15:43:27.928492 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:27.928461    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:27.943157 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:27.943119    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"]
Jul 09 15:43:28.045754 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.045717    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e10546ae-bad0-4125-9760-0cf337001940-podres\") pod \"perf-node-gather-daemonset-dfm2c\" (UID: \"e10546ae-bad0-4125-9760-0cf337001940\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.045954 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.045796    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e10546ae-bad0-4125-9760-0cf337001940-proc\") pod \"perf-node-gather-daemonset-dfm2c\" (UID: \"e10546ae-bad0-4125-9760-0cf337001940\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.046041 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.045985    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxvbx\" (UniqueName: \"kubernetes.io/projected/e10546ae-bad0-4125-9760-0cf337001940-kube-api-access-kxvbx\") pod \"perf-node-gather-daemonset-dfm2c\" (UID: \"e10546ae-bad0-4125-9760-0cf337001940\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.046103 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.046066    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e10546ae-bad0-4125-9760-0cf337001940-sys\") pod \"perf-node-gather-daemonset-dfm2c\" (UID: \"e10546ae-bad0-4125-9760-0cf337001940\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.046103 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.046092    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e10546ae-bad0-4125-9760-0cf337001940-lib-modules\") pod \"perf-node-gather-daemonset-dfm2c\" (UID: \"e10546ae-bad0-4125-9760-0cf337001940\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.147109 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.147053    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e10546ae-bad0-4125-9760-0cf337001940-podres\") pod \"perf-node-gather-daemonset-dfm2c\" (UID: \"e10546ae-bad0-4125-9760-0cf337001940\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.147312 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.147119    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e10546ae-bad0-4125-9760-0cf337001940-proc\") pod \"perf-node-gather-daemonset-dfm2c\" (UID: \"e10546ae-bad0-4125-9760-0cf337001940\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.147312 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.147186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxvbx\" (UniqueName: \"kubernetes.io/projected/e10546ae-bad0-4125-9760-0cf337001940-kube-api-access-kxvbx\") pod \"perf-node-gather-daemonset-dfm2c\" (UID: \"e10546ae-bad0-4125-9760-0cf337001940\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.147312 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.147238    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e10546ae-bad0-4125-9760-0cf337001940-sys\") pod \"perf-node-gather-daemonset-dfm2c\" (UID: \"e10546ae-bad0-4125-9760-0cf337001940\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.147312 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.147261    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e10546ae-bad0-4125-9760-0cf337001940-lib-modules\") pod \"perf-node-gather-daemonset-dfm2c\" (UID: \"e10546ae-bad0-4125-9760-0cf337001940\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.147312 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.147270    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e10546ae-bad0-4125-9760-0cf337001940-proc\") pod \"perf-node-gather-daemonset-dfm2c\" (UID: \"e10546ae-bad0-4125-9760-0cf337001940\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.147312 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.147270    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e10546ae-bad0-4125-9760-0cf337001940-podres\") pod \"perf-node-gather-daemonset-dfm2c\" (UID: \"e10546ae-bad0-4125-9760-0cf337001940\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.147749 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.147322    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e10546ae-bad0-4125-9760-0cf337001940-sys\") pod \"perf-node-gather-daemonset-dfm2c\" (UID: \"e10546ae-bad0-4125-9760-0cf337001940\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.147749 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.147403    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e10546ae-bad0-4125-9760-0cf337001940-lib-modules\") pod \"perf-node-gather-daemonset-dfm2c\" (UID: \"e10546ae-bad0-4125-9760-0cf337001940\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.159578 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.159513    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxvbx\" (UniqueName: \"kubernetes.io/projected/e10546ae-bad0-4125-9760-0cf337001940-kube-api-access-kxvbx\") pod \"perf-node-gather-daemonset-dfm2c\" (UID: \"e10546ae-bad0-4125-9760-0cf337001940\") " pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.240661 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.240572    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:28.436555 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.436510    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"]
Jul 09 15:43:28.440925 ip-10-0-129-152 kubenswrapper[2579]: W0709 15:43:28.440873    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode10546ae_bad0_4125_9760_0cf337001940.slice/crio-ea4dc43cf55569992c02750a0858fce896cad4c50bcbfcce5e8e73a456bf8cf5 WatchSource:0}: Error finding container ea4dc43cf55569992c02750a0858fce896cad4c50bcbfcce5e8e73a456bf8cf5: Status 404 returned error can't find the container with id ea4dc43cf55569992c02750a0858fce896cad4c50bcbfcce5e8e73a456bf8cf5
Jul 09 15:43:28.487842 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:28.487811    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c" event={"ID":"e10546ae-bad0-4125-9760-0cf337001940","Type":"ContainerStarted","Data":"ea4dc43cf55569992c02750a0858fce896cad4c50bcbfcce5e8e73a456bf8cf5"}
Jul 09 15:43:29.496130 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:29.496090    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c" event={"ID":"e10546ae-bad0-4125-9760-0cf337001940","Type":"ContainerStarted","Data":"977da9bce1aef28031d94c9527d12317e7209f130f85e7b1c5a6ad25b97c58fe"}
Jul 09 15:43:29.496624 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:29.496166    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:29.518146 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:29.518071    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c" podStartSLOduration=2.518049811 podStartE2EDuration="2.518049811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:43:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:43:29.516147768 +0000 UTC m=+2459.849577732" watchObservedRunningTime="2026-07-09 15:43:29.518049811 +0000 UTC m=+2459.851479778"
Jul 09 15:43:30.413150 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:30.412556    2579 scope.go:117] "RemoveContainer" containerID="855b84d2cf012e00301f9db99d08d5c89946808428e533dc7a1f056a093155b7"
Jul 09 15:43:31.662027 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:31.661995    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-spbqk_f01b0b67-67ac-416c-9039-da018836dced/dns/0.log"
Jul 09 15:43:31.688746 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:31.688720    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-spbqk_f01b0b67-67ac-416c-9039-da018836dced/kube-rbac-proxy/0.log"
Jul 09 15:43:31.718644 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:31.718565    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jrt9w_7ce9d4e7-f0cf-49b3-b925-12dc259bdd85/dns-node-resolver/0.log"
Jul 09 15:43:32.446014 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:32.445976    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-56b7c85567-zdl9d_1ace1ee5-8e0c-4ff2-9cda-03c238288a9b/registry/0.log"
Jul 09 15:43:32.516271 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:32.516237    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ncj7q_a973d8db-5cd4-490c-af42-c496f3ced07d/node-ca/0.log"
Jul 09 15:43:33.893929 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:33.893890    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b8bb9d8f7-s5tjx_8f8da16e-38d9-48b8-87f0-4925aa46ed7d/kube-auth-proxy/0.log"
Jul 09 15:43:34.085966 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:34.085925    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qhk65_faa2e895-99fe-471d-a028-17843034d623/payload-pre-processing/0.log"
Jul 09 15:43:35.084828 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:35.084795    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q86vm_0817217b-29ab-4c7d-8f05-874d0fc2b557/serve-healthcheck-canary/0.log"
Jul 09 15:43:35.515786 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:35.515526    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-v4z7t/perf-node-gather-daemonset-dfm2c"
Jul 09 15:43:35.831033 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:35.830945    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mfwjc_209f1dc4-865c-46b2-8ee8-83252469d366/kube-rbac-proxy/0.log"
Jul 09 15:43:35.858082 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:35.858055    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mfwjc_209f1dc4-865c-46b2-8ee8-83252469d366/exporter/0.log"
Jul 09 15:43:35.888320 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:35.888287    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mfwjc_209f1dc4-865c-46b2-8ee8-83252469d366/extractor/0.log"
Jul 09 15:43:38.851847 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:38.851802    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jdx44_1c395aa0-45df-428a-aace-f93e3325e9f8/manager/0.log"
Jul 09 15:43:39.011185 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:39.011155    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b8bb9d8f7-s5tjx_8f8da16e-38d9-48b8-87f0-4925aa46ed7d/kube-auth-proxy/0.log"
Jul 09 15:43:39.196531 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:39.196453    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qhk65_faa2e895-99fe-471d-a028-17843034d623/payload-pre-processing/0.log"
Jul 09 15:43:39.199630 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:39.199589    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qdln4_5407f877-73d9-49b5-856b-a880d09f3963/manager/0.log"
Jul 09 15:43:39.330881 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:39.330849    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66b8fbd475-8gj8v_f2917027-3953-4933-b7a4-7b5f7c45d6ef/manager/0.log"
Jul 09 15:43:39.383943 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:39.383919    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66b8fbd475-8gj8v_f2917027-3953-4933-b7a4-7b5f7c45d6ef/copy-manifests/0.log"
Jul 09 15:43:40.745601 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:40.745570    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b8bb9d8f7-s5tjx_8f8da16e-38d9-48b8-87f0-4925aa46ed7d/kube-auth-proxy/0.log"
Jul 09 15:43:40.920831 ip-10-0-129-152 kubenswrapper[2579]: I0709 15:43:40.920793    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qhk65_faa2e895-99fe-471d-a028-17843034d623/payload-pre-processing/0.log"