Sep 07 10:19:55.125867 ip-10-0-128-166 systemd[1]: Starting Kubernetes Kubelet...
Sep 07 10:19:55.554243 ip-10-0-128-166 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 10:19:55.554243 ip-10-0-128-166 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 07 10:19:55.554243 ip-10-0-128-166 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 10:19:55.554243 ip-10-0-128-166 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 07 10:19:55.554243 ip-10-0-128-166 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 07 10:19:55.556823 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.556727    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 07 10:19:55.561055 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561040    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 10:19:55.561055 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561056    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561059    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561062    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561065    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561068    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561071    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561080    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561082    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561085    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561087    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561090    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561093    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561096    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561098    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561101    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561108    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561110    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561113    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561116    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561118    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 10:19:55.561119 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561120    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561123    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561126    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561129    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561131    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561134    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561137    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561140    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561144    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561147    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561150    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561152    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561157    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561160    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561163    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561166    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561170    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561172    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561175    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 10:19:55.561687 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561178    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561181    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561183    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561186    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561188    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561191    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561194    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561196    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561199    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561202    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561204    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561207    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561209    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561212    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561214    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561219    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561222    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561224    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561227    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561229    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 10:19:55.562311 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561231    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561234    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561237    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561239    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561242    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561244    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561247    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561249    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561252    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561254    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561257    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561259    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561263    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561266    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561269    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561272    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561275    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561277    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561280    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561283    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 10:19:55.562851 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561286    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561288    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561291    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561295    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561297    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561299    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561302    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561305    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561308    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561310    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561312    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561316    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561319    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561322    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561324    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561327    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561329    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561331    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 10:19:55.563343 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.561334    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 10:19:55.564488 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564472    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 10:19:55.564488 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564487    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 10:19:55.564488 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564491    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564495    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564499    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564502    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564505    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564508    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564511    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564514    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564517    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564519    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564522    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564524    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564527    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564530    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564533    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564535    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564538    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564541    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564543    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 10:19:55.564645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564563    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564568    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564571    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564574    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564576    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564579    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564582    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564584    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564587    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564590    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564592    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564595    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564597    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564600    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564602    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564606    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564608    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564611    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564613    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564616    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 10:19:55.565110 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564619    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564624    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564628    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564631    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564634    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564638    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564640    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564643    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564647    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564652    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564655    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564658    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564661    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564663    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564666    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564669    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564672    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564675    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564677    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 10:19:55.565678 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564680    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564684    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564687    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564690    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564692    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564695    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564697    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564699    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564704    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564706    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564709    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564712    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564716    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564718    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564721    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564724    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564726    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564729    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564731    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564733    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 10:19:55.566140 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564736    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564738    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564740    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564743    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564746    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564749    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564751    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564754    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564757    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564759    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564761    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564764    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564767    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564770    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564772    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564775    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564777    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564780    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.564782    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565176    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 10:19:55.566658 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565182    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565185    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565188    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565191    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565193    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565196    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565198    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565201    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565204    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565206    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565209    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565211    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565214    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565217    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565219    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565223    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565225    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565228    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565231    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565235    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 10:19:55.567128 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565239    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565242    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565253    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565255    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565259    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565261    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565264    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565267    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565270    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565272    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565275    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565277    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565280    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565283    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565285    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565288    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565290    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565293    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565295    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565297    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 10:19:55.567645 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565300    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565303    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565305    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565307    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565310    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565313    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565315    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565318    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565322    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565325    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565328    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565331    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565334    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565336    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565339    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565342    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565344    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565347    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565350    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 10:19:55.568134 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565352    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565355    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565357    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565360    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565362    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565365    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565367    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565370    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565372    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565374    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565378    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565380    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565383    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565385    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565387    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565390    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565392    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565395    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565397    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565400    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 10:19:55.568613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565402    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565405    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565408    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565410    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565412    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565415    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565418    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565420    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565423    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565425    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565428    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565430    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565434    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565437    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565440    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565442    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565444    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565447    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.565450    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565529    2577 flags.go:64] FLAG: --address="0.0.0.0"
Sep 07 10:19:55.569094 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565582    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565590    2577 flags.go:64] FLAG: --anonymous-auth="true"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565595    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565599    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565603    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565607    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565611    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565615    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565618    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565621    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565625    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565628    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565631    2577 flags.go:64] FLAG: --cgroup-root=""
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565633    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565636    2577 flags.go:64] FLAG: --client-ca-file=""
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565640    2577 flags.go:64] FLAG: --cloud-provider="external"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565643    2577 flags.go:64] FLAG: --cluster-dns="[]"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565647    2577 flags.go:64] FLAG: --cluster-domain=""
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565650    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565653    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565656    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565660    2577 flags.go:64] FLAG: --container-log-max-files="5"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565664    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 07 10:19:55.569571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565666    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565669    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565673    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565675    2577 flags.go:64] FLAG: --contention-profiling="false"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565678    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565681    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565685    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565687    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565692    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565694    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565697    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565700    2577 flags.go:64] FLAG: --enable-load-reader="false"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565703    2577 flags.go:64] FLAG: --enable-server="true"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565706    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565710    2577 flags.go:64] FLAG: --event-burst="100"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565713    2577 flags.go:64] FLAG: --event-qps="50"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565716    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565719    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565722    2577 flags.go:64] FLAG: --eviction-hard=""
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565726    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565729    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565732    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565735    2577 flags.go:64] FLAG: --eviction-soft=""
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565738    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565741    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 07 10:19:55.570117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565744    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565750    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565753    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565755    2577 flags.go:64] FLAG: --fail-swap-on="true"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565758    2577 flags.go:64] FLAG: --feature-gates=""
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565762    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565765    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565768    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565771    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565774    2577 flags.go:64] FLAG: --healthz-port="10248"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565777    2577 flags.go:64] FLAG: --help="false"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565779    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-128-166.ec2.internal"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565782    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565785    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565788    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565791    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565795    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565798    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565800    2577 flags.go:64] FLAG: --image-service-endpoint=""
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565803    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565805    2577 flags.go:64] FLAG: --kube-api-burst="100"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565808    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565811    2577 flags.go:64] FLAG: --kube-api-qps="50"
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565814    2577 flags.go:64] FLAG: --kube-reserved=""
Sep 07 10:19:55.570892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565816    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565819    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565823    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565826    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565829    2577 flags.go:64] FLAG: --lock-file=""
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565832    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565834    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565837    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565843    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565848    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565851    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565854    2577 flags.go:64] FLAG: --logging-format="text"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565857    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565860    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565863    2577 flags.go:64] FLAG: --manifest-url=""
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565866    2577 flags.go:64] FLAG: --manifest-url-header=""
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565870    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565873    2577 flags.go:64] FLAG: --max-open-files="1000000"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565877    2577 flags.go:64] FLAG: --max-pods="110"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565880    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565883    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565886    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565888    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565891    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565894    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 07 10:19:55.571484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565897    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565904    2577 flags.go:64] FLAG: --node-status-max-images="50"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565907    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565910    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565914    2577 flags.go:64] FLAG: --pod-cidr=""
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565916    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565922    2577 flags.go:64] FLAG: --pod-manifest-path=""
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565924    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565927    2577 flags.go:64] FLAG: --pods-per-core="0"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565930    2577 flags.go:64] FLAG: --port="10250"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565935    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565938    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-048cc88d052f1a060"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565941    2577 flags.go:64] FLAG: --qos-reserved=""
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565943    2577 flags.go:64] FLAG: --read-only-port="10255"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565946    2577 flags.go:64] FLAG: --register-node="true"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565949    2577 flags.go:64] FLAG: --register-with-taints=""
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565954    2577 flags.go:64] FLAG: --registry-burst="10"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565957    2577 flags.go:64] FLAG: --registry-qps="5"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565960    2577 flags.go:64] FLAG: --reserved-cpus=""
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565963    2577 flags.go:64] FLAG: --reserved-memory=""
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565966    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565969    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565972    2577 flags.go:64] FLAG: --rotate-certificates="false"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565975    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565977    2577 flags.go:64] FLAG: --runonce="false"
Sep 07 10:19:55.572164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565980    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565983    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565986    2577 flags.go:64] FLAG: --seccomp-default="false"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565988    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565991    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565994    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.565997    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566000    2577 flags.go:64] FLAG: --storage-driver-password="root"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566003    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566005    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566008    2577 flags.go:64] FLAG: --storage-driver-user="root"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566011    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566014    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566016    2577 flags.go:64] FLAG: --system-cgroups=""
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566019    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566024    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566027    2577 flags.go:64] FLAG: --tls-cert-file=""
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566031    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566034    2577 flags.go:64] FLAG: --tls-min-version=""
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566037    2577 flags.go:64] FLAG: --tls-private-key-file=""
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566040    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566042    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566045    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566049    2577 flags.go:64] FLAG: --v="2"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566053    2577 flags.go:64] FLAG: --version="false"
Sep 07 10:19:55.572843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566058    2577 flags.go:64] FLAG: --vmodule=""
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566062    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.566065    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566790    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566796    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566799    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566802    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566805    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566808    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566810    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566813    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566817    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566819    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566822    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566824    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566827    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566830    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566832    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566835    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566837    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 10:19:55.573452 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566839    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566842    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566844    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566847    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566850    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566852    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566855    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566857    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566859    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566862    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566865    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566868    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566870    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566873    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566876    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566879    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566881    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566883    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566886    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566888    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 10:19:55.574013 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566890    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566893    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566895    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566898    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566900    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566903    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566905    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566908    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566910    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566914    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566917    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566920    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566922    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566925    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566927    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566930    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566932    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566935    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566938    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 10:19:55.574518 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566940    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566943    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566945    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566948    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566950    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566953    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566955    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566958    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566960    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566963    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566965    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566968    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566970    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566973    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566975    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566978    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566982    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566985    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566988    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566991    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 10:19:55.575048 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566994    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.566997    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567000    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567002    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567005    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567008    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567010    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567013    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567015    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567018    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567020    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567023    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567026    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567030    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567033    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567035    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567038    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567041    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567044    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 10:19:55.575567 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567047    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 10:19:55.576038 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567050    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 10:19:55.576038 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567052    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 10:19:55.576038 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.567055    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 10:19:55.576038 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.567813    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 10:19:55.576450 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.576428    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 07 10:19:55.576481 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.576451    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 07 10:19:55.576481 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.576471    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 07 10:19:55.576481 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.576477    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 07 10:19:55.576581 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576535    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 10:19:55.576581 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576541    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 10:19:55.576581 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576544    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 10:19:55.576581 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576559    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 10:19:55.576581 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576562    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 10:19:55.576581 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576565    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 10:19:55.576581 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576567    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 10:19:55.576581 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576570    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 10:19:55.576581 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576573    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 10:19:55.576581 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576576    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 10:19:55.576581 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576578    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 10:19:55.576581 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576581    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 10:19:55.576581 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576583    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 10:19:55.576581 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576586    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576589    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576592    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576594    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576597    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576599    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576602    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576604    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576607    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576609    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576613    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576615    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576617    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576621    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576625    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576627    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576639    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576641    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576644    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 10:19:55.576926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576646    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576649    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576652    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576655    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576657    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576660    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576662    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576665    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576667    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576670    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576672    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576675    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576677    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576680    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576682    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576685    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576688    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576691    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576694    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 10:19:55.577385 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576696    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576699    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576702    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576705    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576707    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576710    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576712    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576715    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576718    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576720    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576723    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576727    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576731    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576734    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576737    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576739    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576742    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576744    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576747    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576749    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 10:19:55.577926 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576752    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576754    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576756    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576759    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576762    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576764    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576767    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576769    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576772    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576775    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576779    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576783    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576786    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576788    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576791    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576794    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576796    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576799    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576802    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 10:19:55.578454 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576804    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 10:19:55.578939 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576807    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 10:19:55.578939 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576810    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 10:19:55.578939 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576812    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 10:19:55.578939 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576815    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 10:19:55.578939 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576818    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 10:19:55.578939 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576820    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 10:19:55.578939 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576823    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 10:19:55.578939 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576825    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 10:19:55.578939 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.576830    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 10:19:55.578939 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576946    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 07 10:19:55.578939 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576950    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 07 10:19:55.578939 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576954    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 07 10:19:55.578939 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576957    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 07 10:19:55.578939 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576961    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576964    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576967    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576970    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576974    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576977    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576980    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576983    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576986    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576989    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576992    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576994    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576997    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.576999    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577002    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577004    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577007    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577010    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577012    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 07 10:19:55.579308 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577015    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577017    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577020    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577022    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577025    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577028    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577030    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577033    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577035    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577038    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577040    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577043    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577045    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577048    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577050    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577053    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577055    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577058    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577060    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577063    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 07 10:19:55.579791 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577067    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577070    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577073    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577075    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577077    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577080    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577082    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577085    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577087    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577090    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577092    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577094    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577097    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577099    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577102    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577104    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577106    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577109    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577112    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577114    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 07 10:19:55.580278 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577117    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577119    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577122    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577124    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577126    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577129    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577132    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577134    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577136    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577139    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577142    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577144    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577147    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577150    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577152    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577155    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577157    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577160    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577162    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577165    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 07 10:19:55.580800 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577167    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 07 10:19:55.581279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577170    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 07 10:19:55.581279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577172    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 07 10:19:55.581279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577174    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 07 10:19:55.581279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577177    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 07 10:19:55.581279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577179    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 07 10:19:55.581279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577182    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 07 10:19:55.581279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577185    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 07 10:19:55.581279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577187    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 07 10:19:55.581279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577189    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 07 10:19:55.581279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577193    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 07 10:19:55.581279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577196    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 07 10:19:55.581279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577199    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 07 10:19:55.581279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577201    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 07 10:19:55.581279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577204    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 07 10:19:55.581279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:55.577206    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 07 10:19:55.581661 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.577211    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 07 10:19:55.581661 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.577319    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 07 10:19:55.581661 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.580633    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 07 10:19:55.581760 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.581717    2577 server.go:1019] "Starting client certificate rotation"
Sep 07 10:19:55.581832 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.581815    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 10:19:55.581864 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.581847    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 07 10:19:55.584536 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.584521    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 10:19:55.585900 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.585881    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 07 10:19:55.598316 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.598295    2577 log.go:25] "Validated CRI v1 runtime API"
Sep 07 10:19:55.601819 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.601800    2577 log.go:25] "Validated CRI v1 image API"
Sep 07 10:19:55.603445 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.603431    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 07 10:19:55.606040 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.606006    2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 85f40ada-e99d-4476-b8fe-cda53978619b:/dev/nvme0n1p3 b965e906-6b29-448c-ac04-76f0b9cec8d2:/dev/nvme0n1p4]
Sep 07 10:19:55.606086 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.606040    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 07 10:19:55.608838 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.608820    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 10:19:55.612063 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.611947    2577 manager.go:216] Machine: {Timestamp:2026-09-07 10:19:55.610690737 +0000 UTC m=+0.370446421 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3114801 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec222a564f136e8aa98419a7c79598b0 SystemUUID:ec222a56-4f13-6e8a-a984-19a7c79598b0 BootID:985c3abf-cbdd-4c7c-ac1d-1c6f3b0aba0f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:76:53:4b:d7:4f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:76:53:4b:d7:4f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:6a:92:49:f5:bc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 07 10:19:55.612063 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.612063    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 07 10:19:55.612174 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.612150    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 07 10:19:55.615688 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.615664    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 07 10:19:55.615828 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.615692    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-166.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 07 10:19:55.616559 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.616535    2577 topology_manager.go:138] "Creating topology manager with none policy"
Sep 07 10:19:55.616602 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.616560    2577 container_manager_linux.go:309] "Creating device plugin manager"
Sep 07 10:19:55.616602 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.616574    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 10:19:55.616602 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.616590    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 07 10:19:55.617919 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.617909    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 07 10:19:55.622597 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.622585    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 10:19:55.622731 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.622722    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 07 10:19:55.622761 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.622753    2577 kubelet.go:486] "Attempting to sync node with API server"
Sep 07 10:19:55.622790 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.622763    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 07 10:19:55.622790 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.622782    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 07 10:19:55.622842 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.622792    2577 kubelet.go:398] "Adding apiserver pod source"
Sep 07 10:19:55.622842 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.622801    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 07 10:19:55.623134 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.623123    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 10:19:55.623172 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.623142    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 07 10:19:55.625682 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.625661    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 07 10:19:55.628729 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.628711    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 07 10:19:55.628809 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.628750    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 07 10:19:55.630117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.630101    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 07 10:19:55.630221 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.630123    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 07 10:19:55.630221 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.630132    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 07 10:19:55.630221 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.630141    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 07 10:19:55.630221 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.630153    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 07 10:19:55.630221 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.630162    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 07 10:19:55.630221 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.630180    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 07 10:19:55.630221 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.630188    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 07 10:19:55.630221 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.630197    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 07 10:19:55.630221 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.630205    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 07 10:19:55.630486 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.630227    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 07 10:19:55.630486 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.630236    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 07 10:19:55.631914 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.631902    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 07 10:19:55.631972 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.631919    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 07 10:19:55.635431 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.635409    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-166.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 07 10:19:55.635506 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.635438    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 07 10:19:55.635645 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.635625    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-166.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 07 10:19:55.635946 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.635932    2577 server.go:1268] "Started kubelet"
Sep 07 10:19:55.636118 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.636054    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 07 10:19:55.636177 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.636160    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 07 10:19:55.636219 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.636075    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 07 10:19:55.636910 ip-10-0-128-166 systemd[1]: Started Kubernetes Kubelet.
Sep 07 10:19:55.640709 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.640679    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 07 10:19:55.642286 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.642113    2577 server.go:310] "Adding debug handlers to kubelet server"
Sep 07 10:19:55.644931 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.644909    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 07 10:19:55.647367 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.646156    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-166.ec2.internal.18d302a78c7af7ef  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-166.ec2.internal,UID:ip-10-0-128-166.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-166.ec2.internal,},FirstTimestamp:2026-09-07 10:19:55.635881967 +0000 UTC m=+0.395637651,LastTimestamp:2026-09-07 10:19:55.635881967 +0000 UTC m=+0.395637651,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-166.ec2.internal,}"
Sep 07 10:19:55.652107 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.652092    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 07 10:19:55.652460 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.652444    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 07 10:19:55.654119 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.654106    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-166.ec2.internal\" not found"
Sep 07 10:19:55.654771 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.654761    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 07 10:19:55.654771 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.654771    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 07 10:19:55.654945 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.654928    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 07 10:19:55.655052 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.654970    2577 reconstruct.go:97] "Volume reconstruction finished"
Sep 07 10:19:55.655052 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.655051    2577 reconciler.go:29] "Reconciler: start to sync state"
Sep 07 10:19:55.655998 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.655983    2577 factory.go:153] Registering CRI-O factory
Sep 07 10:19:55.655998 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.655999    2577 factory.go:223] Registration of the crio container factory successfully
Sep 07 10:19:55.656109 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.656038    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 07 10:19:55.656109 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.656044    2577 factory.go:55] Registering systemd factory
Sep 07 10:19:55.656109 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.656050    2577 factory.go:223] Registration of the systemd container factory successfully
Sep 07 10:19:55.656109 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.656065    2577 factory.go:103] Registering Raw factory
Sep 07 10:19:55.656109 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.656075    2577 manager.go:1196] Started watching for new ooms in manager
Sep 07 10:19:55.656427 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.656413    2577 manager.go:349] Starting recovery of all containers
Sep 07 10:19:55.659715 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.659693    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qgnqm"
Sep 07 10:19:55.665354 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.665334    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qgnqm"
Sep 07 10:19:55.666538 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.666504    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-166.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 07 10:19:55.666714 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.666669    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 07 10:19:55.667988 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.667975    2577 manager.go:354] Recovery completed
Sep 07 10:19:55.672872 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.672860    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 10:19:55.674721 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.674705    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 10:19:55.674786 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.674733    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 10:19:55.674786 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.674743    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasSufficientPID"
Sep 07 10:19:55.675292 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.675280    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 07 10:19:55.675292 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.675290    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 07 10:19:55.675417 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.675305    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 07 10:19:55.675417 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.675335    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 10:19:55.676104 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.676090    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 10:19:55.676187 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.676126    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 10:19:55.676187 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.676141    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasSufficientPID"
Sep 07 10:19:55.677315 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.677303    2577 policy_none.go:49] "None policy: Start"
Sep 07 10:19:55.677373 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.677319    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 07 10:19:55.677373 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.677329    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 07 10:19:55.679352 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.679342    2577 policy_none.go:47] "Start"
Sep 07 10:19:55.728640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.721160    2577 manager.go:337] "Starting Device Plugin manager"
Sep 07 10:19:55.728640 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.721205    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 07 10:19:55.728640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.721216    2577 server.go:87] "Starting device plugin registration server"
Sep 07 10:19:55.728640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.721580    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 07 10:19:55.728640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.721592    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 07 10:19:55.728640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.721695    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 07 10:19:55.728640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.721791    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 07 10:19:55.728640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.721799    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 07 10:19:55.728640 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.722318    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 07 10:19:55.728640 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.722364    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-166.ec2.internal\" not found"
Sep 07 10:19:55.750951 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.750925    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 07 10:19:55.752199 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.752186    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 07 10:19:55.752263 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.752232    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 07 10:19:55.752263 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.752256    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 07 10:19:55.752361 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.752297    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 07 10:19:55.754258 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.754235    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 10:19:55.822417 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.822299    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 10:19:55.823195 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.823179    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 10:19:55.823276 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.823208    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 10:19:55.823276 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.823220    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasSufficientPID"
Sep 07 10:19:55.823276 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.823244    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-166.ec2.internal"
Sep 07 10:19:55.831987 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.831967    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-166.ec2.internal"
Sep 07 10:19:55.831987 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.831989    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-166.ec2.internal\": node \"ip-10-0-128-166.ec2.internal\" not found"
Sep 07 10:19:55.853302 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.853277    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-166.ec2.internal"]
Sep 07 10:19:55.853391 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.853342    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 10:19:55.854580 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.854561    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-166.ec2.internal\" not found"
Sep 07 10:19:55.854761 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.854747    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 10:19:55.854825 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.854776    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 10:19:55.854825 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.854789    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasSufficientPID"
Sep 07 10:19:55.855179 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.855161    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/52d6397be1be19265439934d092e7c0f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal\" (UID: \"52d6397be1be19265439934d092e7c0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal"
Sep 07 10:19:55.855214 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.855186    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52d6397be1be19265439934d092e7c0f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal\" (UID: \"52d6397be1be19265439934d092e7c0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal"
Sep 07 10:19:55.857004 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.856993    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 10:19:55.857128 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.857114    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal"
Sep 07 10:19:55.857177 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.857165    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 10:19:55.857741 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.857723    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 10:19:55.857824 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.857752    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 10:19:55.857824 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.857759    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 10:19:55.857824 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.857767    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasSufficientPID"
Sep 07 10:19:55.857824 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.857778    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 10:19:55.857824 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.857787    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasSufficientPID"
Sep 07 10:19:55.859992 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.859976    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-166.ec2.internal"
Sep 07 10:19:55.860066 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.860009    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 07 10:19:55.860733 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.860716    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasSufficientMemory"
Sep 07 10:19:55.860812 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.860746    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasNoDiskPressure"
Sep 07 10:19:55.860812 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.860758    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeHasSufficientPID"
Sep 07 10:19:55.889285 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.889263    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-166.ec2.internal\" not found" node="ip-10-0-128-166.ec2.internal"
Sep 07 10:19:55.893694 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.893679    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-166.ec2.internal\" not found" node="ip-10-0-128-166.ec2.internal"
Sep 07 10:19:55.955379 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:55.955346    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-166.ec2.internal\" not found"
Sep 07 10:19:55.955521 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.955395    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/52d6397be1be19265439934d092e7c0f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal\" (UID: \"52d6397be1be19265439934d092e7c0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal"
Sep 07 10:19:55.955521 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.955425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52d6397be1be19265439934d092e7c0f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal\" (UID: \"52d6397be1be19265439934d092e7c0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal"
Sep 07 10:19:55.955521 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.955475    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52d6397be1be19265439934d092e7c0f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal\" (UID: \"52d6397be1be19265439934d092e7c0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal"
Sep 07 10:19:55.955521 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:55.955487    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/52d6397be1be19265439934d092e7c0f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal\" (UID: \"52d6397be1be19265439934d092e7c0f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal"
Sep 07 10:19:56.055650 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:56.055614    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-166.ec2.internal\" not found"
Sep 07 10:19:56.055650 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.055645    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/91bd6355dd5f19af696dfdda483965bf-config\") pod \"kube-apiserver-proxy-ip-10-0-128-166.ec2.internal\" (UID: \"91bd6355dd5f19af696dfdda483965bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-166.ec2.internal"
Sep 07 10:19:56.156250 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:56.156181    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-166.ec2.internal\" not found"
Sep 07 10:19:56.156250 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.156219    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/91bd6355dd5f19af696dfdda483965bf-config\") pod \"kube-apiserver-proxy-ip-10-0-128-166.ec2.internal\" (UID: \"91bd6355dd5f19af696dfdda483965bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-166.ec2.internal"
Sep 07 10:19:56.156363 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.156275    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/91bd6355dd5f19af696dfdda483965bf-config\") pod \"kube-apiserver-proxy-ip-10-0-128-166.ec2.internal\" (UID: \"91bd6355dd5f19af696dfdda483965bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-166.ec2.internal"
Sep 07 10:19:56.191398 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.191354    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal"
Sep 07 10:19:56.197268 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.197252    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-166.ec2.internal"
Sep 07 10:19:56.256531 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:56.256496    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-166.ec2.internal\" not found"
Sep 07 10:19:56.356910 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:56.356887    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-166.ec2.internal\" not found"
Sep 07 10:19:56.457269 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:56.457184    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-166.ec2.internal\" not found"
Sep 07 10:19:56.542535 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.542506    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 10:19:56.557361 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:56.557336    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-166.ec2.internal\" not found"
Sep 07 10:19:56.581527 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.581503    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 07 10:19:56.581691 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.581671    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 07 10:19:56.581771 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.581687    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 07 10:19:56.658431 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:56.658403    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-166.ec2.internal\" not found"
Sep 07 10:19:56.671684 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.671652    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-06 10:14:56 +0000 UTC" deadline="2028-05-25 08:46:01.537075236 +0000 UTC"
Sep 07 10:19:56.671684 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.671681    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15022h26m4.865397475s"
Sep 07 10:19:56.746853 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:56.746803    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91bd6355dd5f19af696dfdda483965bf.slice/crio-d3a10e618fd80f5afcb5a3f5d62118510d48207b40db1a535babedd67e59f7b2 WatchSource:0}: Error finding container d3a10e618fd80f5afcb5a3f5d62118510d48207b40db1a535babedd67e59f7b2: Status 404 returned error can't find the container with id d3a10e618fd80f5afcb5a3f5d62118510d48207b40db1a535babedd67e59f7b2
Sep 07 10:19:56.747109 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:56.747085    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52d6397be1be19265439934d092e7c0f.slice/crio-5e9f43f43fd118e4f1b4f75e076b89e85f310803da9fc5f0aec4cfe3668f11cb WatchSource:0}: Error finding container 5e9f43f43fd118e4f1b4f75e076b89e85f310803da9fc5f0aec4cfe3668f11cb: Status 404 returned error can't find the container with id 5e9f43f43fd118e4f1b4f75e076b89e85f310803da9fc5f0aec4cfe3668f11cb
Sep 07 10:19:56.750427 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.750396    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 10:19:56.755905 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.755865    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal" event={"ID":"52d6397be1be19265439934d092e7c0f","Type":"ContainerStarted","Data":"5e9f43f43fd118e4f1b4f75e076b89e85f310803da9fc5f0aec4cfe3668f11cb"}
Sep 07 10:19:56.756912 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.756892    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-166.ec2.internal" event={"ID":"91bd6355dd5f19af696dfdda483965bf","Type":"ContainerStarted","Data":"d3a10e618fd80f5afcb5a3f5d62118510d48207b40db1a535babedd67e59f7b2"}
Sep 07 10:19:56.759075 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:56.759060    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-166.ec2.internal\" not found"
Sep 07 10:19:56.767739 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.767723    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 10:19:56.855809 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.855775    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-166.ec2.internal"
Sep 07 10:19:56.865992 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.865972    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 10:19:56.867849 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.867830    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal"
Sep 07 10:19:56.879509 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:56.879484    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 07 10:19:57.019537 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:57.019449    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 10:19:58.649724 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.649696    2577 apiserver.go:52] "Watching apiserver"
Sep 07 10:19:58.663299 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.663089    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 07 10:19:58.665193 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.665161    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-166.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal","openshift-multus/network-metrics-daemon-x8w2s","openshift-network-operator/iptables-alerter-vq7pn","openshift-ovn-kubernetes/ovnkube-node-m4b2f","kube-system/global-pull-secret-syncer-ktvqp","kube-system/konnectivity-agent-7kpgf","openshift-cluster-node-tuning-operator/tuned-h8ptk","openshift-dns/node-resolver-4hg64","openshift-image-registry/node-ca-mljbz","openshift-multus/multus-additional-cni-plugins-hp27c","openshift-multus/multus-pjkq5","openshift-network-diagnostics/network-check-target-9xlk4"]
Sep 07 10:19:58.667617 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.667590    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vq7pn"
Sep 07 10:19:58.670848 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.670673    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 07 10:19:58.670848 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.670740    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bj6mq\""
Sep 07 10:19:58.670848 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.670792    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 07 10:19:58.670848 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.670673    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 07 10:19:58.671446 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.671225    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4hg64"
Sep 07 10:19:58.673658 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.673640    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 07 10:19:58.673658 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.673663    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-v7cms\""
Sep 07 10:19:58.673817 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.673667    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 07 10:19:58.673914 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.673897    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.676013 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.675992    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 07 10:19:58.676367 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.676349    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sb4v5\""
Sep 07 10:19:58.677027 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.676556    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 07 10:19:58.678682 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.678287    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7kpgf"
Sep 07 10:19:58.678682 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.678425    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:19:58.678682 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:58.678491    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:19:58.680218 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.680201    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 07 10:19:58.680464 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.680441    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 07 10:19:58.680554 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.680473    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4zj8j\""
Sep 07 10:19:58.680687 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.680668    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 07 10:19:58.680748 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.680738    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.682879 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.682862    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:19:58.682969 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:58.682928    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:19:58.683295 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.683277    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 07 10:19:58.683574 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.683557    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 07 10:19:58.683910 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.683557    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 07 10:19:58.683910 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.683688    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 07 10:19:58.684415 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.684395    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 07 10:19:58.684415 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.684411    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 07 10:19:58.684657 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.684641    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4gjq4\""
Sep 07 10:19:58.685783 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.685767    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.688065 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.687695    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mljbz"
Sep 07 10:19:58.688065 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.687921    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 07 10:19:58.688419 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.688364    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2dt68\""
Sep 07 10:19:58.688419 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.688382    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 07 10:19:58.688555 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.688365    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 07 10:19:58.689966 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.689945    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jhk84\""
Sep 07 10:19:58.690053 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.689981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-systemd-units\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.690053 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-cni-netd\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.690053 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690034    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6ce1c919-010d-441e-adb2-8c356a98d262-iptables-alerter-script\") pod \"iptables-alerter-vq7pn\" (UID: \"6ce1c919-010d-441e-adb2-8c356a98d262\") " pod="openshift-network-operator/iptables-alerter-vq7pn"
Sep 07 10:19:58.690183 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690059    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ce1c919-010d-441e-adb2-8c356a98d262-host-slash\") pod \"iptables-alerter-vq7pn\" (UID: \"6ce1c919-010d-441e-adb2-8c356a98d262\") " pod="openshift-network-operator/iptables-alerter-vq7pn"
Sep 07 10:19:58.690183 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690082    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-sysctl-conf\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.690183 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-systemd\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.690183 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690115    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 07 10:19:58.690183 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690158    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 07 10:19:58.690391 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690125    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-run\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.690391 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/804db42a-4910-4e6f-bdd2-d62cbb720877-dbus\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:19:58.690391 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690232    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 07 10:19:58.690391 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690243    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:19:58.690391 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690267    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0187e151-799f-402c-9886-6cb9b0bba10d-hosts-file\") pod \"node-resolver-4hg64\" (UID: \"0187e151-799f-402c-9886-6cb9b0bba10d\") " pod="openshift-dns/node-resolver-4hg64"
Sep 07 10:19:58.690391 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690289    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-lib-modules\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.690391 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztsh5\" (UniqueName: \"kubernetes.io/projected/dccff851-014f-42e2-8d03-2892d4baf2c9-kube-api-access-ztsh5\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.690391 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690342    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0187e151-799f-402c-9886-6cb9b0bba10d-tmp-dir\") pod \"node-resolver-4hg64\" (UID: \"0187e151-799f-402c-9886-6cb9b0bba10d\") " pod="openshift-dns/node-resolver-4hg64"
Sep 07 10:19:58.690391 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-kubelet\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.690678 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-run-systemd\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.690678 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690454    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-var-lib-openvswitch\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.690678 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690479    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-etc-openvswitch\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.690678 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690503    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-run-openvswitch\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.690678 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690554    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.690935 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690551    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-sysconfig\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.690935 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690880    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dccff851-014f-42e2-8d03-2892d4baf2c9-tmp\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.690935 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690910    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f838808a-c1be-438f-b71b-b9148e6ec777-agent-certs\") pod \"konnectivity-agent-7kpgf\" (UID: \"f838808a-c1be-438f-b71b-b9148e6ec777\") " pod="kube-system/konnectivity-agent-7kpgf"
Sep 07 10:19:58.691040 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690956    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-run-ovn-kubernetes\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.691040 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.690985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.691040 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.691012    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ee29c7da-349c-4e72-b3bf-3dacc671b261-ovnkube-config\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.691203 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.691038    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee29c7da-349c-4e72-b3bf-3dacc671b261-ovn-node-metrics-cert\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.691203 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.691068    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfgm7\" (UniqueName: \"kubernetes.io/projected/415363e6-edcf-41e5-97ce-cd379cf30f4e-kube-api-access-mfgm7\") pod \"network-metrics-daemon-x8w2s\" (UID: \"415363e6-edcf-41e5-97ce-cd379cf30f4e\") " pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:19:58.691203 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.691187    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-run-ovn\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.691339 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.691211    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-node-log\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.691339 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.691235    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-log-socket\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.691339 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.691258    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ee29c7da-349c-4e72-b3bf-3dacc671b261-env-overrides\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.691339 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.691303    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbl5q\" (UniqueName: \"kubernetes.io/projected/ee29c7da-349c-4e72-b3bf-3dacc671b261-kube-api-access-kbl5q\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.691339 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.691331    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-var-lib-kubelet\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.691563 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.691371    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs\") pod \"network-metrics-daemon-x8w2s\" (UID: \"415363e6-edcf-41e5-97ce-cd379cf30f4e\") " pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:19:58.691563 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.691420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ee29c7da-349c-4e72-b3bf-3dacc671b261-ovnkube-script-lib\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.691563 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.691465    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdgmq\" (UniqueName: \"kubernetes.io/projected/6ce1c919-010d-441e-adb2-8c356a98d262-kube-api-access-tdgmq\") pod \"iptables-alerter-vq7pn\" (UID: \"6ce1c919-010d-441e-adb2-8c356a98d262\") " pod="openshift-network-operator/iptables-alerter-vq7pn"
Sep 07 10:19:58.692080 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.692053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-sysctl-d\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.692195 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.692102    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-host\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.692195 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.692149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-modprobe-d\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.692195 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.692174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-kubernetes\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.692347 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.692219    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/804db42a-4910-4e6f-bdd2-d62cbb720877-kubelet-config\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:19:58.692347 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.692247    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv4rx\" (UniqueName: \"kubernetes.io/projected/0187e151-799f-402c-9886-6cb9b0bba10d-kube-api-access-kv4rx\") pod \"node-resolver-4hg64\" (UID: \"0187e151-799f-402c-9886-6cb9b0bba10d\") " pod="openshift-dns/node-resolver-4hg64"
Sep 07 10:19:58.692347 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.692273    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-slash\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.692347 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.692325    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-run-netns\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.692528 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.692379    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-cni-bin\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.692528 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.692405    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-sys\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.692528 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.692428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-tuned\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.692528 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.692452    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f838808a-c1be-438f-b71b-b9148e6ec777-konnectivity-ca\") pod \"konnectivity-agent-7kpgf\" (UID: \"f838808a-c1be-438f-b71b-b9148e6ec777\") " pod="kube-system/konnectivity-agent-7kpgf"
Sep 07 10:19:58.692715 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.692627    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 07 10:19:58.693167 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.693028    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.694699 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.693764    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-d9b8l\""
Sep 07 10:19:58.694699 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.694022    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 07 10:19:58.694699 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.694165    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 07 10:19:58.694699 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.694350    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 07 10:19:58.694699 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.694405    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 07 10:19:58.695089 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.694822    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 07 10:19:58.695405 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.695384    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fk67j\""
Sep 07 10:19:58.695512 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.695494    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 07 10:19:58.695892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.695872    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:19:58.695978 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:58.695957    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:19:58.714818 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.714794    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xq6pp"
Sep 07 10:19:58.720046 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.720025    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xq6pp"
Sep 07 10:19:58.781377 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.781352    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 07 10:19:58.793273 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793242    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-var-lib-kubelet\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.793444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793288    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-var-lib-cni-multus\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.793444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793329    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0e183d73-394f-4794-a848-d217bb88c3fa-multus-daemon-config\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.793444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793356    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdgmq\" (UniqueName: \"kubernetes.io/projected/6ce1c919-010d-441e-adb2-8c356a98d262-kube-api-access-tdgmq\") pod \"iptables-alerter-vq7pn\" (UID: \"6ce1c919-010d-441e-adb2-8c356a98d262\") " pod="openshift-network-operator/iptables-alerter-vq7pn"
Sep 07 10:19:58.793444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793371    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-var-lib-kubelet\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.793444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793427    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-host\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.793444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-host\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.793710 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793473    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bcc4fa7a-3956-4a04-8fc7-b1914d6079f8-serviceca\") pod \"node-ca-mljbz\" (UID: \"bcc4fa7a-3956-4a04-8fc7-b1914d6079f8\") " pod="openshift-image-registry/node-ca-mljbz"
Sep 07 10:19:58.793710 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793499    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-device-dir\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.793710 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793523    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0e183d73-394f-4794-a848-d217bb88c3fa-cni-binary-copy\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.793710 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793546    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-hostroot\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.793710 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-modprobe-d\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.793710 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-run-netns\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.793710 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793617    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-run-multus-certs\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.793710 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793639    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqzzn\" (UniqueName: \"kubernetes.io/projected/0e183d73-394f-4794-a848-d217bb88c3fa-kube-api-access-dqzzn\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.793710 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-systemd-units\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.793710 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793686    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntfrb\" (UniqueName: \"kubernetes.io/projected/bcc4fa7a-3956-4a04-8fc7-b1914d6079f8-kube-api-access-ntfrb\") pod \"node-ca-mljbz\" (UID: \"bcc4fa7a-3956-4a04-8fc7-b1914d6079f8\") " pod="openshift-image-registry/node-ca-mljbz"
Sep 07 10:19:58.793710 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793711    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ac75d0ea-397b-457f-b8b1-46735f0dff26-cnibin\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.794192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793824    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-modprobe-d\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.794192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793851    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ac75d0ea-397b-457f-b8b1-46735f0dff26-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.794192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793879    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-run\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.794192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793899    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/804db42a-4910-4e6f-bdd2-d62cbb720877-dbus\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:19:58.794192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:19:58.794192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.793956    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-run-systemd\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.794192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-multus-socket-dir-parent\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.794192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-lib-modules\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.794192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794059    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztsh5\" (UniqueName: \"kubernetes.io/projected/dccff851-014f-42e2-8d03-2892d4baf2c9-kube-api-access-ztsh5\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.794192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794086    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-registration-dir\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.794192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794101    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkppv\" (UniqueName: \"kubernetes.io/projected/aac01bef-b7e3-4a1b-8862-b956e3054b68-kube-api-access-fkppv\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.794192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794123    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-run-netns\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.794192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794160    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-var-lib-kubelet\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.794192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794184    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-multus-conf-dir\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-sysconfig\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dccff851-014f-42e2-8d03-2892d4baf2c9-tmp\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794351    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f838808a-c1be-438f-b71b-b9148e6ec777-agent-certs\") pod \"konnectivity-agent-7kpgf\" (UID: \"f838808a-c1be-438f-b71b-b9148e6ec777\") " pod="kube-system/konnectivity-agent-7kpgf"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-multus-cni-dir\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794410    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-run-k8s-cni-cncf-io\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfgm7\" (UniqueName: \"kubernetes.io/projected/415363e6-edcf-41e5-97ce-cd379cf30f4e-kube-api-access-mfgm7\") pod \"network-metrics-daemon-x8w2s\" (UID: \"415363e6-edcf-41e5-97ce-cd379cf30f4e\") " pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794457    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-run-ovn\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs\") pod \"network-metrics-daemon-x8w2s\" (UID: \"415363e6-edcf-41e5-97ce-cd379cf30f4e\") " pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ee29c7da-349c-4e72-b3bf-3dacc671b261-ovnkube-script-lib\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794501    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bcc4fa7a-3956-4a04-8fc7-b1914d6079f8-host\") pod \"node-ca-mljbz\" (UID: \"bcc4fa7a-3956-4a04-8fc7-b1914d6079f8\") " pod="openshift-image-registry/node-ca-mljbz"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794517    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ac75d0ea-397b-457f-b8b1-46735f0dff26-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794538    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmgnd\" (UniqueName: \"kubernetes.io/projected/ac75d0ea-397b-457f-b8b1-46735f0dff26-kube-api-access-fmgnd\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794559    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-system-cni-dir\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794575    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-sysctl-d\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-kubernetes\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794611    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/804db42a-4910-4e6f-bdd2-d62cbb720877-kubelet-config\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:19:58.794679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794635    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv4rx\" (UniqueName: \"kubernetes.io/projected/0187e151-799f-402c-9886-6cb9b0bba10d-kube-api-access-kv4rx\") pod \"node-resolver-4hg64\" (UID: \"0187e151-799f-402c-9886-6cb9b0bba10d\") " pod="openshift-dns/node-resolver-4hg64"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794653    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-slash\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794666    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-cni-bin\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794682    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ac75d0ea-397b-457f-b8b1-46735f0dff26-cni-binary-copy\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794699    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794796    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-sys\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-tuned\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f838808a-c1be-438f-b71b-b9148e6ec777-konnectivity-ca\") pod \"konnectivity-agent-7kpgf\" (UID: \"f838808a-c1be-438f-b71b-b9148e6ec777\") " pod="kube-system/konnectivity-agent-7kpgf"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-cni-netd\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794869    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ac75d0ea-397b-457f-b8b1-46735f0dff26-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-sys-fs\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794915    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6ce1c919-010d-441e-adb2-8c356a98d262-iptables-alerter-script\") pod \"iptables-alerter-vq7pn\" (UID: \"6ce1c919-010d-441e-adb2-8c356a98d262\") " pod="openshift-network-operator/iptables-alerter-vq7pn"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794940    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ce1c919-010d-441e-adb2-8c356a98d262-host-slash\") pod \"iptables-alerter-vq7pn\" (UID: \"6ce1c919-010d-441e-adb2-8c356a98d262\") " pod="openshift-network-operator/iptables-alerter-vq7pn"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794964    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-sysctl-conf\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.794991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-systemd\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795015    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0187e151-799f-402c-9886-6cb9b0bba10d-hosts-file\") pod \"node-resolver-4hg64\" (UID: \"0187e151-799f-402c-9886-6cb9b0bba10d\") " pod="openshift-dns/node-resolver-4hg64"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795039    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-kubelet\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.795298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-etc-openvswitch\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795087    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-run-openvswitch\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795110    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0187e151-799f-402c-9886-6cb9b0bba10d-tmp-dir\") pod \"node-resolver-4hg64\" (UID: \"0187e151-799f-402c-9886-6cb9b0bba10d\") " pod="openshift-dns/node-resolver-4hg64"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-var-lib-openvswitch\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795175    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795200    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-node-log\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795223    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ee29c7da-349c-4e72-b3bf-3dacc671b261-env-overrides\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795249    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-cnibin\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795272    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-run-ovn-kubernetes\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ee29c7da-349c-4e72-b3bf-3dacc671b261-ovnkube-config\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795321    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee29c7da-349c-4e72-b3bf-3dacc671b261-ovn-node-metrics-cert\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795347    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac75d0ea-397b-457f-b8b1-46735f0dff26-system-cni-dir\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795371    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-etc-selinux\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795395    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-var-lib-cni-bin\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795423    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-log-socket\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbl5q\" (UniqueName: \"kubernetes.io/projected/ee29c7da-349c-4e72-b3bf-3dacc671b261-kube-api-access-kbl5q\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795475    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-socket-dir\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.795820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-etc-kubernetes\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795521    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jczdv\" (UniqueName: \"kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv\") pod \"network-check-target-9xlk4\" (UID: \"e537c01e-65d3-4e1f-ab15-84ddc4b0dc46\") " pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795870    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-run-systemd\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795901    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-run-openvswitch\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795941    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-run-netns\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795961    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-slash\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.795982    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-systemd-units\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.796004    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-cni-bin\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.796028    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-run\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.796060    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-sys\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.796077    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/804db42a-4910-4e6f-bdd2-d62cbb720877-dbus\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.796090    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-sysconfig\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:58.796182    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.796255    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:58.796285    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret podName:804db42a-4910-4e6f-bdd2-d62cbb720877 nodeName:}" failed. No retries permitted until 2026-09-07 10:19:59.296250727 +0000 UTC m=+3.030424954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret") pod "global-pull-secret-syncer-ktvqp" (UID: "804db42a-4910-4e6f-bdd2-d62cbb720877") : object "kube-system"/"original-pull-secret" not registered
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.796335    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-run-ovn\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:58.796399    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 10:19:58.796613 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:58.796443    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs podName:415363e6-edcf-41e5-97ce-cd379cf30f4e nodeName:}" failed. No retries permitted until 2026-09-07 10:19:59.29642729 +0000 UTC m=+3.030601515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs") pod "network-metrics-daemon-x8w2s" (UID: "415363e6-edcf-41e5-97ce-cd379cf30f4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 10:19:58.797507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.796520    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-lib-modules\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.797507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.796585    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ce1c919-010d-441e-adb2-8c356a98d262-host-slash\") pod \"iptables-alerter-vq7pn\" (UID: \"6ce1c919-010d-441e-adb2-8c356a98d262\") " pod="openshift-network-operator/iptables-alerter-vq7pn"
Sep 07 10:19:58.797507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.796634    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-cni-netd\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.797507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.796770    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f838808a-c1be-438f-b71b-b9148e6ec777-konnectivity-ca\") pod \"konnectivity-agent-7kpgf\" (UID: \"f838808a-c1be-438f-b71b-b9148e6ec777\") " pod="kube-system/konnectivity-agent-7kpgf"
Sep 07 10:19:58.797507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.796895    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-sysctl-conf\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.797507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.796932    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-systemd\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.797507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.796976    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0187e151-799f-402c-9886-6cb9b0bba10d-hosts-file\") pod \"node-resolver-4hg64\" (UID: \"0187e151-799f-402c-9886-6cb9b0bba10d\") " pod="openshift-dns/node-resolver-4hg64"
Sep 07 10:19:58.797507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.797008    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-kubelet\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.797507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.797035    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-etc-openvswitch\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.797507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.797220    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-node-log\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.797507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.797307    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ee29c7da-349c-4e72-b3bf-3dacc671b261-ovnkube-script-lib\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.797507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.797355    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6ce1c919-010d-441e-adb2-8c356a98d262-iptables-alerter-script\") pod \"iptables-alerter-vq7pn\" (UID: \"6ce1c919-010d-441e-adb2-8c356a98d262\") " pod="openshift-network-operator/iptables-alerter-vq7pn"
Sep 07 10:19:58.797507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.797425    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-sysctl-d\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.797507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.797501    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0187e151-799f-402c-9886-6cb9b0bba10d-tmp-dir\") pod \"node-resolver-4hg64\" (UID: \"0187e151-799f-402c-9886-6cb9b0bba10d\") " pod="openshift-dns/node-resolver-4hg64"
Sep 07 10:19:58.798279 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.797606    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ee29c7da-349c-4e72-b3bf-3dacc671b261-ovnkube-config\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.798279 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.797618    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-kubernetes\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.798279 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.797669    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/804db42a-4910-4e6f-bdd2-d62cbb720877-kubelet-config\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:19:58.798279 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.797705    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-run-ovn-kubernetes\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.798279 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.797720    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ee29c7da-349c-4e72-b3bf-3dacc671b261-env-overrides\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.798279 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.797744    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-var-lib-openvswitch\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.798279 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.797768    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.798279 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.797807    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ee29c7da-349c-4e72-b3bf-3dacc671b261-log-socket\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.800788 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.800585    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dccff851-014f-42e2-8d03-2892d4baf2c9-etc-tuned\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.800948 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.800817    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f838808a-c1be-438f-b71b-b9148e6ec777-agent-certs\") pod \"konnectivity-agent-7kpgf\" (UID: \"f838808a-c1be-438f-b71b-b9148e6ec777\") " pod="kube-system/konnectivity-agent-7kpgf"
Sep 07 10:19:58.801018 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.800994    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee29c7da-349c-4e72-b3bf-3dacc671b261-ovn-node-metrics-cert\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.801018 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.801008    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dccff851-014f-42e2-8d03-2892d4baf2c9-tmp\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.804553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.804527    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdgmq\" (UniqueName: \"kubernetes.io/projected/6ce1c919-010d-441e-adb2-8c356a98d262-kube-api-access-tdgmq\") pod \"iptables-alerter-vq7pn\" (UID: \"6ce1c919-010d-441e-adb2-8c356a98d262\") " pod="openshift-network-operator/iptables-alerter-vq7pn"
Sep 07 10:19:58.809308 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.809279    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfgm7\" (UniqueName: \"kubernetes.io/projected/415363e6-edcf-41e5-97ce-cd379cf30f4e-kube-api-access-mfgm7\") pod \"network-metrics-daemon-x8w2s\" (UID: \"415363e6-edcf-41e5-97ce-cd379cf30f4e\") " pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:19:58.809669 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.809645    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbl5q\" (UniqueName: \"kubernetes.io/projected/ee29c7da-349c-4e72-b3bf-3dacc671b261-kube-api-access-kbl5q\") pod \"ovnkube-node-m4b2f\" (UID: \"ee29c7da-349c-4e72-b3bf-3dacc671b261\") " pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:58.810185 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.810165    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv4rx\" (UniqueName: \"kubernetes.io/projected/0187e151-799f-402c-9886-6cb9b0bba10d-kube-api-access-kv4rx\") pod \"node-resolver-4hg64\" (UID: \"0187e151-799f-402c-9886-6cb9b0bba10d\") " pod="openshift-dns/node-resolver-4hg64"
Sep 07 10:19:58.811061 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.811043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztsh5\" (UniqueName: \"kubernetes.io/projected/dccff851-014f-42e2-8d03-2892d4baf2c9-kube-api-access-ztsh5\") pod \"tuned-h8ptk\" (UID: \"dccff851-014f-42e2-8d03-2892d4baf2c9\") " pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:58.896823 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.896789    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bcc4fa7a-3956-4a04-8fc7-b1914d6079f8-serviceca\") pod \"node-ca-mljbz\" (UID: \"bcc4fa7a-3956-4a04-8fc7-b1914d6079f8\") " pod="openshift-image-registry/node-ca-mljbz"
Sep 07 10:19:58.896984 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.896831    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-device-dir\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.897042 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.896985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0e183d73-394f-4794-a848-d217bb88c3fa-cni-binary-copy\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.897075 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897056    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-device-dir\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.897164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897125    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-hostroot\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.897244 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-run-multus-certs\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.897244 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897192    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-hostroot\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.897244 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqzzn\" (UniqueName: \"kubernetes.io/projected/0e183d73-394f-4794-a848-d217bb88c3fa-kube-api-access-dqzzn\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.897244 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897219    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntfrb\" (UniqueName: \"kubernetes.io/projected/bcc4fa7a-3956-4a04-8fc7-b1914d6079f8-kube-api-access-ntfrb\") pod \"node-ca-mljbz\" (UID: \"bcc4fa7a-3956-4a04-8fc7-b1914d6079f8\") " pod="openshift-image-registry/node-ca-mljbz"
Sep 07 10:19:58.897244 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897224    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-run-multus-certs\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.897244 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897243    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ac75d0ea-397b-457f-b8b1-46735f0dff26-cnibin\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.897490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897259    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ac75d0ea-397b-457f-b8b1-46735f0dff26-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.897490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-multus-socket-dir-parent\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.897490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-registration-dir\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.897490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897328    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkppv\" (UniqueName: \"kubernetes.io/projected/aac01bef-b7e3-4a1b-8862-b956e3054b68-kube-api-access-fkppv\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.897490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897328    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ac75d0ea-397b-457f-b8b1-46735f0dff26-cnibin\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.897490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897348    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-run-netns\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.897490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-var-lib-kubelet\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.897490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897395    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-multus-conf-dir\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.897490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897400    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bcc4fa7a-3956-4a04-8fc7-b1914d6079f8-serviceca\") pod \"node-ca-mljbz\" (UID: \"bcc4fa7a-3956-4a04-8fc7-b1914d6079f8\") " pod="openshift-image-registry/node-ca-mljbz"
Sep 07 10:19:58.897490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897420    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-multus-cni-dir\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.897490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-run-k8s-cni-cncf-io\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.897490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897475    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-var-lib-kubelet\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.897490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897480    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bcc4fa7a-3956-4a04-8fc7-b1914d6079f8-host\") pod \"node-ca-mljbz\" (UID: \"bcc4fa7a-3956-4a04-8fc7-b1914d6079f8\") " pod="openshift-image-registry/node-ca-mljbz"
Sep 07 10:19:58.897490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897483    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ac75d0ea-397b-457f-b8b1-46735f0dff26-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897403    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-run-netns\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897486    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-multus-socket-dir-parent\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897509    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ac75d0ea-397b-457f-b8b1-46735f0dff26-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897531    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-multus-conf-dir\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897537    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmgnd\" (UniqueName: \"kubernetes.io/projected/ac75d0ea-397b-457f-b8b1-46735f0dff26-kube-api-access-fmgnd\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897539    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-run-k8s-cni-cncf-io\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897547    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-multus-cni-dir\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897557    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-registration-dir\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897571    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-system-cni-dir\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897583    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bcc4fa7a-3956-4a04-8fc7-b1914d6079f8-host\") pod \"node-ca-mljbz\" (UID: \"bcc4fa7a-3956-4a04-8fc7-b1914d6079f8\") " pod="openshift-image-registry/node-ca-mljbz"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ac75d0ea-397b-457f-b8b1-46735f0dff26-cni-binary-copy\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897632    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ac75d0ea-397b-457f-b8b1-46735f0dff26-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897682    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-system-cni-dir\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-sys-fs\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897736    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-sys-fs\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.898155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-cnibin\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac75d0ea-397b-457f-b8b1-46735f0dff26-system-cni-dir\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897802    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-etc-selinux\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897777    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-cnibin\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-var-lib-cni-bin\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897865    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-var-lib-cni-bin\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897746    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-socket-dir\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897909    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-etc-kubernetes\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jczdv\" (UniqueName: \"kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv\") pod \"network-check-target-9xlk4\" (UID: \"e537c01e-65d3-4e1f-ab15-84ddc4b0dc46\") " pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897935    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-etc-selinux\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897961    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-var-lib-cni-multus\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897974    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aac01bef-b7e3-4a1b-8862-b956e3054b68-socket-dir\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897981    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac75d0ea-397b-457f-b8b1-46735f0dff26-system-cni-dir\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.897986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0e183d73-394f-4794-a848-d217bb88c3fa-multus-daemon-config\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.898022    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-host-var-lib-cni-multus\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.898056    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e183d73-394f-4794-a848-d217bb88c3fa-etc-kubernetes\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.898797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.898122    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ac75d0ea-397b-457f-b8b1-46735f0dff26-cni-binary-copy\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.899290 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.898282    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ac75d0ea-397b-457f-b8b1-46735f0dff26-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.899290 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.898302    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ac75d0ea-397b-457f-b8b1-46735f0dff26-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.899290 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.898506    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0e183d73-394f-4794-a848-d217bb88c3fa-cni-binary-copy\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.899290 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.898525    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0e183d73-394f-4794-a848-d217bb88c3fa-multus-daemon-config\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.903264 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:58.903196    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 10:19:58.903264 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:58.903222    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 10:19:58.903264 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:58.903235    2577 projected.go:196] Error preparing data for projected volume kube-api-access-jczdv for pod openshift-network-diagnostics/network-check-target-9xlk4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 10:19:58.903452 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:58.903292    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv podName:e537c01e-65d3-4e1f-ab15-84ddc4b0dc46 nodeName:}" failed. No retries permitted until 2026-09-07 10:19:59.403276056 +0000 UTC m=+3.137450290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jczdv" (UniqueName: "kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv") pod "network-check-target-9xlk4" (UID: "e537c01e-65d3-4e1f-ab15-84ddc4b0dc46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 10:19:58.905829 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.905808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmgnd\" (UniqueName: \"kubernetes.io/projected/ac75d0ea-397b-457f-b8b1-46735f0dff26-kube-api-access-fmgnd\") pod \"multus-additional-cni-plugins-hp27c\" (UID: \"ac75d0ea-397b-457f-b8b1-46735f0dff26\") " pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:58.905970 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.905948    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkppv\" (UniqueName: \"kubernetes.io/projected/aac01bef-b7e3-4a1b-8862-b956e3054b68-kube-api-access-fkppv\") pod \"aws-ebs-csi-driver-node-4hkrd\" (UID: \"aac01bef-b7e3-4a1b-8862-b956e3054b68\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:58.906766 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.906746    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntfrb\" (UniqueName: \"kubernetes.io/projected/bcc4fa7a-3956-4a04-8fc7-b1914d6079f8-kube-api-access-ntfrb\") pod \"node-ca-mljbz\" (UID: \"bcc4fa7a-3956-4a04-8fc7-b1914d6079f8\") " pod="openshift-image-registry/node-ca-mljbz"
Sep 07 10:19:58.906850 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.906756    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqzzn\" (UniqueName: \"kubernetes.io/projected/0e183d73-394f-4794-a848-d217bb88c3fa-kube-api-access-dqzzn\") pod \"multus-pjkq5\" (UID: \"0e183d73-394f-4794-a848-d217bb88c3fa\") " pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:58.982890 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.982853    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vq7pn"
Sep 07 10:19:58.988080 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.988062    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4hg64"
Sep 07 10:19:58.999002 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:58.998982    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h8ptk"
Sep 07 10:19:59.004354 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:59.004317    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7kpgf"
Sep 07 10:19:59.012419 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:59.012395    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:19:59.012563 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:59.012532    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 10:19:59.022202 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:59.022178    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd"
Sep 07 10:19:59.031971 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:59.031951    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mljbz"
Sep 07 10:19:59.038634 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:59.038613    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hp27c"
Sep 07 10:19:59.044244 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:59.044227    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pjkq5"
Sep 07 10:19:59.301539 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:59.301453    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs\") pod \"network-metrics-daemon-x8w2s\" (UID: \"415363e6-edcf-41e5-97ce-cd379cf30f4e\") " pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:19:59.301539 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:59.301519    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:19:59.301755 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:59.301628    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 10:19:59.301755 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:59.301630    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 10:19:59.301755 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:59.301689    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret podName:804db42a-4910-4e6f-bdd2-d62cbb720877 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:00.301674983 +0000 UTC m=+4.035849194 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret") pod "global-pull-secret-syncer-ktvqp" (UID: "804db42a-4910-4e6f-bdd2-d62cbb720877") : object "kube-system"/"original-pull-secret" not registered
Sep 07 10:19:59.301755 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:59.301703    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs podName:415363e6-edcf-41e5-97ce-cd379cf30f4e nodeName:}" failed. No retries permitted until 2026-09-07 10:20:00.301696858 +0000 UTC m=+4.035871069 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs") pod "network-metrics-daemon-x8w2s" (UID: "415363e6-edcf-41e5-97ce-cd379cf30f4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 10:19:59.503046 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:59.503021    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jczdv\" (UniqueName: \"kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv\") pod \"network-check-target-9xlk4\" (UID: \"e537c01e-65d3-4e1f-ab15-84ddc4b0dc46\") " pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:19:59.503179 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:59.503164    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 10:19:59.503236 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:59.503185    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 10:19:59.503236 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:59.503194    2577 projected.go:196] Error preparing data for projected volume kube-api-access-jczdv for pod openshift-network-diagnostics/network-check-target-9xlk4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 10:19:59.503316 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:59.503241    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv podName:e537c01e-65d3-4e1f-ab15-84ddc4b0dc46 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:00.50322759 +0000 UTC m=+4.237401815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jczdv" (UniqueName: "kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv") pod "network-check-target-9xlk4" (UID: "e537c01e-65d3-4e1f-ab15-84ddc4b0dc46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 10:19:59.724059 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:59.723988    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 10:14:58 +0000 UTC" deadline="2028-04-27 01:18:48.483868163 +0000 UTC"
Sep 07 10:19:59.724059 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:59.724016    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14342h58m48.759855621s"
Sep 07 10:19:59.778445 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:59.778414    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:19:59.778636 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:59.778527    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:19:59.778636 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:19:59.778584    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:19:59.778713 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:19:59.778684    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:19:59.822365 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:59.822331    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee29c7da_349c_4e72_b3bf_3dacc671b261.slice/crio-46e50af9c160b73c841b185c20fe68bc3404e7718d3eb86c01a6c94135bfdce4 WatchSource:0}: Error finding container 46e50af9c160b73c841b185c20fe68bc3404e7718d3eb86c01a6c94135bfdce4: Status 404 returned error can't find the container with id 46e50af9c160b73c841b185c20fe68bc3404e7718d3eb86c01a6c94135bfdce4
Sep 07 10:19:59.823909 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:59.823883    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0187e151_799f_402c_9886_6cb9b0bba10d.slice/crio-7c8e8221845091c4359ea77a0336c80640aa7819a77fa228b85623b49aff0026 WatchSource:0}: Error finding container 7c8e8221845091c4359ea77a0336c80640aa7819a77fa228b85623b49aff0026: Status 404 returned error can't find the container with id 7c8e8221845091c4359ea77a0336c80640aa7819a77fa228b85623b49aff0026
Sep 07 10:19:59.832122 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:59.832101    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaac01bef_b7e3_4a1b_8862_b956e3054b68.slice/crio-f57f681b27cb33ce9682ac30592f612640a5b3ac8905ea3d97b3464385e01209 WatchSource:0}: Error finding container f57f681b27cb33ce9682ac30592f612640a5b3ac8905ea3d97b3464385e01209: Status 404 returned error can't find the container with id f57f681b27cb33ce9682ac30592f612640a5b3ac8905ea3d97b3464385e01209
Sep 07 10:19:59.833647 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:59.833629    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac75d0ea_397b_457f_b8b1_46735f0dff26.slice/crio-b07c62895b0f8f4aedfa4bb457b0c77e47bd12748f1774c8af0e3853fac6aa93 WatchSource:0}: Error finding container b07c62895b0f8f4aedfa4bb457b0c77e47bd12748f1774c8af0e3853fac6aa93: Status 404 returned error can't find the container with id b07c62895b0f8f4aedfa4bb457b0c77e47bd12748f1774c8af0e3853fac6aa93
Sep 07 10:19:59.835779 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:59.835678    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ce1c919_010d_441e_adb2_8c356a98d262.slice/crio-6f697a92162ebe494b93f0bb52fcb441f6c3a17179df559754800952a3e257f1 WatchSource:0}: Error finding container 6f697a92162ebe494b93f0bb52fcb441f6c3a17179df559754800952a3e257f1: Status 404 returned error can't find the container with id 6f697a92162ebe494b93f0bb52fcb441f6c3a17179df559754800952a3e257f1
Sep 07 10:19:59.848329 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:19:59.848301    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e183d73_394f_4794_a848_d217bb88c3fa.slice/crio-095d0594a1b5bdcdc294ad88b63d1f49ba6109a666d2000a1c5af74a392629a6 WatchSource:0}: Error finding container 095d0594a1b5bdcdc294ad88b63d1f49ba6109a666d2000a1c5af74a392629a6: Status 404 returned error can't find the container with id 095d0594a1b5bdcdc294ad88b63d1f49ba6109a666d2000a1c5af74a392629a6
Sep 07 10:20:00.309392 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.309346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:00.309539 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.309405    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs\") pod \"network-metrics-daemon-x8w2s\" (UID: \"415363e6-edcf-41e5-97ce-cd379cf30f4e\") " pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:00.309539 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:00.309515    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 10:20:00.309661 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:00.309586    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret podName:804db42a-4910-4e6f-bdd2-d62cbb720877 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:02.309571104 +0000 UTC m=+6.043745315 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret") pod "global-pull-secret-syncer-ktvqp" (UID: "804db42a-4910-4e6f-bdd2-d62cbb720877") : object "kube-system"/"original-pull-secret" not registered
Sep 07 10:20:00.309661 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:00.309520    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 10:20:00.309769 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:00.309681    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs podName:415363e6-edcf-41e5-97ce-cd379cf30f4e nodeName:}" failed. No retries permitted until 2026-09-07 10:20:02.309663912 +0000 UTC m=+6.043838142 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs") pod "network-metrics-daemon-x8w2s" (UID: "415363e6-edcf-41e5-97ce-cd379cf30f4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 10:20:00.510962 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.510922    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jczdv\" (UniqueName: \"kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv\") pod \"network-check-target-9xlk4\" (UID: \"e537c01e-65d3-4e1f-ab15-84ddc4b0dc46\") " pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:00.511166 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:00.511147    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 10:20:00.511231 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:00.511177    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 10:20:00.511231 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:00.511190    2577 projected.go:196] Error preparing data for projected volume kube-api-access-jczdv for pod openshift-network-diagnostics/network-check-target-9xlk4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 10:20:00.511350 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:00.511252    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv podName:e537c01e-65d3-4e1f-ab15-84ddc4b0dc46 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:02.511233216 +0000 UTC m=+6.245407441 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jczdv" (UniqueName: "kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv") pod "network-check-target-9xlk4" (UID: "e537c01e-65d3-4e1f-ab15-84ddc4b0dc46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 10:20:00.679508 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.679429    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 07 10:20:00.724787 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.724723    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-06 10:14:58 +0000 UTC" deadline="2028-04-21 08:19:45.507087723 +0000 UTC"
Sep 07 10:20:00.724787 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.724786    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14205h59m44.782306129s"
Sep 07 10:20:00.780546 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.780517    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:00.780753 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:00.780727    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:00.798000 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.797965    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7kpgf" event={"ID":"f838808a-c1be-438f-b71b-b9148e6ec777","Type":"ContainerStarted","Data":"53e70c51a54c4ed784c2a8559f639a9de3f54592bd28d03425b0b40b063d739d"}
Sep 07 10:20:00.818169 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.818120    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vq7pn" event={"ID":"6ce1c919-010d-441e-adb2-8c356a98d262","Type":"ContainerStarted","Data":"6f697a92162ebe494b93f0bb52fcb441f6c3a17179df559754800952a3e257f1"}
Sep 07 10:20:00.844048 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.843945    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd" event={"ID":"aac01bef-b7e3-4a1b-8862-b956e3054b68","Type":"ContainerStarted","Data":"f57f681b27cb33ce9682ac30592f612640a5b3ac8905ea3d97b3464385e01209"}
Sep 07 10:20:00.848381 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.848304    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4hg64" event={"ID":"0187e151-799f-402c-9886-6cb9b0bba10d","Type":"ContainerStarted","Data":"7c8e8221845091c4359ea77a0336c80640aa7819a77fa228b85623b49aff0026"}
Sep 07 10:20:00.868273 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.868191    2577 generic.go:356] "Generic (PLEG): container finished" podID="52d6397be1be19265439934d092e7c0f" containerID="2d42a7b1f342f05de67a702a3c6d9f697664934d39c5b6c52891a342eabc7eb5" exitCode=0
Sep 07 10:20:00.868430 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.868338    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal" event={"ID":"52d6397be1be19265439934d092e7c0f","Type":"ContainerDied","Data":"2d42a7b1f342f05de67a702a3c6d9f697664934d39c5b6c52891a342eabc7eb5"}
Sep 07 10:20:00.886102 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.886072    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mljbz" event={"ID":"bcc4fa7a-3956-4a04-8fc7-b1914d6079f8","Type":"ContainerStarted","Data":"2d1062afe3ecc8403d0cf8e7990e0850c3dcb96f884978f473e40426eafb76c6"}
Sep 07 10:20:00.895875 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.895839    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f" event={"ID":"ee29c7da-349c-4e72-b3bf-3dacc671b261","Type":"ContainerStarted","Data":"46e50af9c160b73c841b185c20fe68bc3404e7718d3eb86c01a6c94135bfdce4"}
Sep 07 10:20:00.911437 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.911378    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-166.ec2.internal" event={"ID":"91bd6355dd5f19af696dfdda483965bf","Type":"ContainerStarted","Data":"b23a384754809873629d94c116df25552b1e86a0af5dba6d634538ff42baf32a"}
Sep 07 10:20:00.925549 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.925515    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pjkq5" event={"ID":"0e183d73-394f-4794-a848-d217bb88c3fa","Type":"ContainerStarted","Data":"095d0594a1b5bdcdc294ad88b63d1f49ba6109a666d2000a1c5af74a392629a6"}
Sep 07 10:20:00.939597 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.939531    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hp27c" event={"ID":"ac75d0ea-397b-457f-b8b1-46735f0dff26","Type":"ContainerStarted","Data":"b07c62895b0f8f4aedfa4bb457b0c77e47bd12748f1774c8af0e3853fac6aa93"}
Sep 07 10:20:00.961308 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:00.961274    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h8ptk" event={"ID":"dccff851-014f-42e2-8d03-2892d4baf2c9","Type":"ContainerStarted","Data":"8d9060b80f7a5d984c1651b4b2b5beafae33cbffbe827d249ec3eb1242968e32"}
Sep 07 10:20:01.778325 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:01.778290    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:01.778791 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:01.778295    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:01.778791 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:01.778434    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:01.778791 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:01.778511    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:01.993529 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:01.993461    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal" event={"ID":"52d6397be1be19265439934d092e7c0f","Type":"ContainerStarted","Data":"bac61f5ba8dc0300c161dae5569d80bc62e3a249a8aeb36adb4316ccb515d70d"}
Sep 07 10:20:02.008662 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:02.007976    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-166.ec2.internal" podStartSLOduration=5.007952925 podStartE2EDuration="5.007952925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:00.925357145 +0000 UTC m=+4.659531378" watchObservedRunningTime="2026-09-07 10:20:02.007952925 +0000 UTC m=+5.742127158"
Sep 07 10:20:02.331192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:02.330884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:02.331369 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:02.331251    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs\") pod \"network-metrics-daemon-x8w2s\" (UID: \"415363e6-edcf-41e5-97ce-cd379cf30f4e\") " pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:02.331446 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:02.331402    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 10:20:02.331511 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:02.331462    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs podName:415363e6-edcf-41e5-97ce-cd379cf30f4e nodeName:}" failed. No retries permitted until 2026-09-07 10:20:06.331444239 +0000 UTC m=+10.065618465 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs") pod "network-metrics-daemon-x8w2s" (UID: "415363e6-edcf-41e5-97ce-cd379cf30f4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 10:20:02.331993 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:02.331973    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 10:20:02.332110 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:02.332029    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret podName:804db42a-4910-4e6f-bdd2-d62cbb720877 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:06.33201432 +0000 UTC m=+10.066188537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret") pod "global-pull-secret-syncer-ktvqp" (UID: "804db42a-4910-4e6f-bdd2-d62cbb720877") : object "kube-system"/"original-pull-secret" not registered
Sep 07 10:20:02.533646 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:02.532973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jczdv\" (UniqueName: \"kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv\") pod \"network-check-target-9xlk4\" (UID: \"e537c01e-65d3-4e1f-ab15-84ddc4b0dc46\") " pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:02.533646 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:02.533191    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 10:20:02.533646 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:02.533210    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 10:20:02.533646 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:02.533223    2577 projected.go:196] Error preparing data for projected volume kube-api-access-jczdv for pod openshift-network-diagnostics/network-check-target-9xlk4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 10:20:02.533646 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:02.533282    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv podName:e537c01e-65d3-4e1f-ab15-84ddc4b0dc46 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:06.533264562 +0000 UTC m=+10.267438796 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jczdv" (UniqueName: "kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv") pod "network-check-target-9xlk4" (UID: "e537c01e-65d3-4e1f-ab15-84ddc4b0dc46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 10:20:02.786054 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:02.785315    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:02.786054 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:02.785507    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:03.778887 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:03.778846    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:03.779052 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:03.778985    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:03.779377 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:03.779362    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:03.779450 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:03.779431    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:04.779334 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:04.779291    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:04.779793 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:04.779422    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:05.779097 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:05.778980    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:05.779276 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:05.779111    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:05.779526 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:05.779502    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:05.779889 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:05.779619    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:06.367611 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:06.367573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:06.367799 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:06.367645    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs\") pod \"network-metrics-daemon-x8w2s\" (UID: \"415363e6-edcf-41e5-97ce-cd379cf30f4e\") " pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:06.367799 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:06.367741    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 10:20:06.367799 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:06.367784    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 10:20:06.367963 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:06.367817    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret podName:804db42a-4910-4e6f-bdd2-d62cbb720877 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:14.367797067 +0000 UTC m=+18.101971291 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret") pod "global-pull-secret-syncer-ktvqp" (UID: "804db42a-4910-4e6f-bdd2-d62cbb720877") : object "kube-system"/"original-pull-secret" not registered
Sep 07 10:20:06.367963 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:06.367845    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs podName:415363e6-edcf-41e5-97ce-cd379cf30f4e nodeName:}" failed. No retries permitted until 2026-09-07 10:20:14.36782635 +0000 UTC m=+18.102000565 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs") pod "network-metrics-daemon-x8w2s" (UID: "415363e6-edcf-41e5-97ce-cd379cf30f4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 10:20:06.568983 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:06.568942    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jczdv\" (UniqueName: \"kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv\") pod \"network-check-target-9xlk4\" (UID: \"e537c01e-65d3-4e1f-ab15-84ddc4b0dc46\") " pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:06.569217 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:06.569075    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 10:20:06.569217 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:06.569093    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 10:20:06.569217 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:06.569105    2577 projected.go:196] Error preparing data for projected volume kube-api-access-jczdv for pod openshift-network-diagnostics/network-check-target-9xlk4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 10:20:06.569217 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:06.569171    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv podName:e537c01e-65d3-4e1f-ab15-84ddc4b0dc46 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:14.569155247 +0000 UTC m=+18.303329459 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jczdv" (UniqueName: "kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv") pod "network-check-target-9xlk4" (UID: "e537c01e-65d3-4e1f-ab15-84ddc4b0dc46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 10:20:06.780111 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:06.778494    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:06.780111 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:06.778634    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:07.778571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:07.778023    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:07.778571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:07.778071    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:07.778571 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:07.778183    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:07.778571 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:07.778325    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:08.779016 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:08.778978    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:08.779468 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:08.779151    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:09.778786 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:09.778749    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:09.778976 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:09.778756    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:09.778976 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:09.778888    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:09.778976 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:09.778965    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:10.780345 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:10.780172    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:10.780890 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:10.780850    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:11.778613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:11.778573    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:11.778803 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:11.778573    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:11.778803 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:11.778705    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:11.778803 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:11.778775    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:12.779011 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:12.778977    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:12.779484 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:12.779124    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:13.778103 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:13.778066    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:13.778103 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:13.778111    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:13.778380 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:13.778214    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:13.778380 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:13.778319    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:14.428588 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:14.428549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:14.429065 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:14.428613    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs\") pod \"network-metrics-daemon-x8w2s\" (UID: \"415363e6-edcf-41e5-97ce-cd379cf30f4e\") " pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:14.429065 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:14.428704    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 10:20:14.429065 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:14.428779    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret podName:804db42a-4910-4e6f-bdd2-d62cbb720877 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:30.428762373 +0000 UTC m=+34.162936584 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret") pod "global-pull-secret-syncer-ktvqp" (UID: "804db42a-4910-4e6f-bdd2-d62cbb720877") : object "kube-system"/"original-pull-secret" not registered
Sep 07 10:20:14.429065 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:14.428781    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 10:20:14.429065 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:14.428842    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs podName:415363e6-edcf-41e5-97ce-cd379cf30f4e nodeName:}" failed. No retries permitted until 2026-09-07 10:20:30.428823498 +0000 UTC m=+34.162997724 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs") pod "network-metrics-daemon-x8w2s" (UID: "415363e6-edcf-41e5-97ce-cd379cf30f4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 10:20:14.630509 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:14.630433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jczdv\" (UniqueName: \"kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv\") pod \"network-check-target-9xlk4\" (UID: \"e537c01e-65d3-4e1f-ab15-84ddc4b0dc46\") " pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:14.630680 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:14.630606    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 10:20:14.630680 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:14.630628    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 10:20:14.630680 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:14.630641    2577 projected.go:196] Error preparing data for projected volume kube-api-access-jczdv for pod openshift-network-diagnostics/network-check-target-9xlk4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 10:20:14.630817 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:14.630703    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv podName:e537c01e-65d3-4e1f-ab15-84ddc4b0dc46 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:30.630687981 +0000 UTC m=+34.364862216 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jczdv" (UniqueName: "kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv") pod "network-check-target-9xlk4" (UID: "e537c01e-65d3-4e1f-ab15-84ddc4b0dc46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 10:20:14.778557 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:14.778473    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:14.778711 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:14.778631    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:15.778665 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:15.778631    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:15.779124 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:15.778630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:15.779124 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:15.778768    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:15.779124 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:15.778860    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:16.779442 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:16.779404    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:16.779903 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:16.779529    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:17.778352 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:17.778308    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:17.778550 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:17.778315    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:17.778550 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:17.778419    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:17.778550 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:17.778512    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:18.779182 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:18.778963    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:18.779827 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:18.779304    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:19.035056 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.035023    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mljbz" event={"ID":"bcc4fa7a-3956-4a04-8fc7-b1914d6079f8","Type":"ContainerStarted","Data":"7b7566d6e4c05befb334152b8dbe46c4789ae4c9900d6f7b24fcafd9ebdc9e75"}
Sep 07 10:20:19.038304 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.038273    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f" event={"ID":"ee29c7da-349c-4e72-b3bf-3dacc671b261","Type":"ContainerStarted","Data":"bbe941376e01b33791a0c1cee590221ae7f7b42b591a81d60dd838e81e60b3b2"}
Sep 07 10:20:19.038444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.038345    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f" event={"ID":"ee29c7da-349c-4e72-b3bf-3dacc671b261","Type":"ContainerStarted","Data":"643923cba176b4fbab0119aea96a4a0ff2ff8717f55de0dc24db1f5e51a34eef"}
Sep 07 10:20:19.038444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.038394    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f" event={"ID":"ee29c7da-349c-4e72-b3bf-3dacc671b261","Type":"ContainerStarted","Data":"e1ca9340087c187bd33e31c69f68f68819e9e620005b1a0b3e3e95f1c813fe5e"}
Sep 07 10:20:19.038444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.038428    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f" event={"ID":"ee29c7da-349c-4e72-b3bf-3dacc671b261","Type":"ContainerStarted","Data":"421d2f1d2f1afc18e4699893b0576e0981689f9deea85e512b06d2359d7f337e"}
Sep 07 10:20:19.038609 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.038477    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f" event={"ID":"ee29c7da-349c-4e72-b3bf-3dacc671b261","Type":"ContainerStarted","Data":"15584f80811583159d03469e40709a96ff8a3ac8ead4f5cc1a4abd621c79bcf4"}
Sep 07 10:20:19.038609 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.038515    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f" event={"ID":"ee29c7da-349c-4e72-b3bf-3dacc671b261","Type":"ContainerStarted","Data":"6eda068274b4d02b5ef1f94327964d7aab10ebea03db88e328c273cdb1ba5b89"}
Sep 07 10:20:19.039726 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.039694    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pjkq5" event={"ID":"0e183d73-394f-4794-a848-d217bb88c3fa","Type":"ContainerStarted","Data":"68b569d157038a67a29d711f04e31eb5830d60a0a6bd296e60a70a8101037214"}
Sep 07 10:20:19.041057 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.041032    2577 generic.go:356] "Generic (PLEG): container finished" podID="ac75d0ea-397b-457f-b8b1-46735f0dff26" containerID="0445c08a8916381f85d28ce89b08e4372f61fae52878c36bdfe8c9424e6d6124" exitCode=0
Sep 07 10:20:19.041242 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.041097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hp27c" event={"ID":"ac75d0ea-397b-457f-b8b1-46735f0dff26","Type":"ContainerDied","Data":"0445c08a8916381f85d28ce89b08e4372f61fae52878c36bdfe8c9424e6d6124"}
Sep 07 10:20:19.042668 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.042589    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h8ptk" event={"ID":"dccff851-014f-42e2-8d03-2892d4baf2c9","Type":"ContainerStarted","Data":"049b6eff0756bbb0a03ec96da6a5d9ced78ff7464d99124ef1a1d36631e95b0f"}
Sep 07 10:20:19.044201 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.044178    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7kpgf" event={"ID":"f838808a-c1be-438f-b71b-b9148e6ec777","Type":"ContainerStarted","Data":"b4aecbc3a3b2dd46cdd7bcfc480c04aa2b3a38da457634eb84ad589cad6f8ee9"}
Sep 07 10:20:19.045636 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.045616    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd" event={"ID":"aac01bef-b7e3-4a1b-8862-b956e3054b68","Type":"ContainerStarted","Data":"227cab75a0807aaa75fd699d3218500e3105dc1ece00b1346a738e82223fa8d7"}
Sep 07 10:20:19.049010 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.048989    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4hg64" event={"ID":"0187e151-799f-402c-9886-6cb9b0bba10d","Type":"ContainerStarted","Data":"396e66390e1d8172cfb2082d0bc31bd48476eb9e9a116379e7ef3285abc7b1b4"}
Sep 07 10:20:19.052244 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.052210    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mljbz" podStartSLOduration=3.909462254 podStartE2EDuration="22.052197909s" totalImagesPullingTime="18.142735655s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:19.052126524 +0000 UTC m=+22.786300756" watchObservedRunningTime="2026-09-07 10:20:19.052197909 +0000 UTC m=+22.786372141"
Sep 07 10:20:19.052531 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.052508    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-166.ec2.internal" podStartSLOduration=22.052502185 podStartE2EDuration="22.052502185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:02.008643545 +0000 UTC m=+5.742817779" watchObservedRunningTime="2026-09-07 10:20:19.052502185 +0000 UTC m=+22.786676418"
Sep 07 10:20:19.067315 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.067269    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pjkq5" podStartSLOduration=3.894346545 podStartE2EDuration="22.067257052s" totalImagesPullingTime="18.172910507s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:19.067178594 +0000 UTC m=+22.801352824" watchObservedRunningTime="2026-09-07 10:20:19.067257052 +0000 UTC m=+22.801431345"
Sep 07 10:20:19.082659 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.082617    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4hg64" podStartSLOduration=4.928511883 podStartE2EDuration="23.082603318s" totalImagesPullingTime="18.154091435s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:19:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:19.082065418 +0000 UTC m=+22.816239650" watchObservedRunningTime="2026-09-07 10:20:19.082603318 +0000 UTC m=+22.816777550"
Sep 07 10:20:19.099195 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.099126    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h8ptk" podStartSLOduration=4.956885685 podStartE2EDuration="23.099110018s" totalImagesPullingTime="18.142224333s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:19:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:19.09883084 +0000 UTC m=+22.833005072" watchObservedRunningTime="2026-09-07 10:20:19.099110018 +0000 UTC m=+22.833284251"
Sep 07 10:20:19.113325 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.113281    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7kpgf" podStartSLOduration=9.105272116 podStartE2EDuration="23.113265218s" totalImagesPullingTime="14.007993102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:19:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:19.113174424 +0000 UTC m=+22.847348657" watchObservedRunningTime="2026-09-07 10:20:19.113265218 +0000 UTC m=+22.847439448"
Sep 07 10:20:19.567556 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.567403    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 07 10:20:19.757765 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.757666    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-07T10:20:19.567551562Z","UUID":"3bb43b9d-e256-4340-b595-5ada873f33a8","Handler":null,"Name":"","Endpoint":""}
Sep 07 10:20:19.760566 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.760543    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 07 10:20:19.760566 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.760574    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 07 10:20:19.778069 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.778041    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:19.778069 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:19.778068    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:19.778316 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:19.778173    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:19.778316 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:19.778239    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:20.053243 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:20.053157    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vq7pn" event={"ID":"6ce1c919-010d-441e-adb2-8c356a98d262","Type":"ContainerStarted","Data":"e0759f77261cfd562f7905a66b0bec594e8c4a89b8de4ab7c4fde986190333d7"}
Sep 07 10:20:20.057618 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:20.057584    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd" event={"ID":"aac01bef-b7e3-4a1b-8862-b956e3054b68","Type":"ContainerStarted","Data":"b82bc15242c028a7e65dc36b5fc848be2a5795fa3ebb1b121b6ea3d51b1abe18"}
Sep 07 10:20:20.778652 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:20.778628    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:20.778808 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:20.778739    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:21.062860 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:21.062820    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd" event={"ID":"aac01bef-b7e3-4a1b-8862-b956e3054b68","Type":"ContainerStarted","Data":"c03e16eb74f1339b6ce67e7440367cae646a3b67068c0447c1864c4402686358"}
Sep 07 10:20:21.068337 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:21.068291    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f" event={"ID":"ee29c7da-349c-4e72-b3bf-3dacc671b261","Type":"ContainerStarted","Data":"7b8b997592b5a6b45827c675322e98a0edaea6015666e0b0762fa51e6839f5bd"}
Sep 07 10:20:21.079821 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:21.079762    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hkrd" podStartSLOduration=3.567542638 podStartE2EDuration="24.07974116s" totalImagesPullingTime="20.512198522s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:19:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:21.079584448 +0000 UTC m=+24.813758682" watchObservedRunningTime="2026-09-07 10:20:21.07974116 +0000 UTC m=+24.813915396"
Sep 07 10:20:21.080047 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:21.080018    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vq7pn" podStartSLOduration=6.934406211 podStartE2EDuration="25.080009115s" totalImagesPullingTime="18.145602904s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:19:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:20.067871403 +0000 UTC m=+23.802045636" watchObservedRunningTime="2026-09-07 10:20:21.080009115 +0000 UTC m=+24.814183351"
Sep 07 10:20:21.779093 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:21.779053    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:21.779093 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:21.779074    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:21.779322 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:21.779188    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:21.779322 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:21.779270    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:21.901193 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:21.901156    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7kpgf"
Sep 07 10:20:21.901781 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:21.901753    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7kpgf"
Sep 07 10:20:22.071385 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:22.071297    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7kpgf"
Sep 07 10:20:22.071799 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:22.071551    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7kpgf"
Sep 07 10:20:22.778708 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:22.778669    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:22.778936 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:22.778818    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:23.778808 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:23.778772    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:23.778808 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:23.778805    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:23.779365 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:23.778919    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:23.779365 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:23.779058    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:24.080303 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:24.079479    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f" event={"ID":"ee29c7da-349c-4e72-b3bf-3dacc671b261","Type":"ContainerStarted","Data":"695a77c139fca1eb7dae3a9e42c921fe7e38659d64c37b45c02d078cd1b12aec"}
Sep 07 10:20:24.080303 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:24.079761    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:20:24.080303 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:24.079806    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:20:24.080303 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:24.079851    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:20:24.096878 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:24.096851    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:20:24.102630 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:24.102606    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:20:24.103640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:24.103598    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f" podStartSLOduration=8.886941471 podStartE2EDuration="27.103585166s" totalImagesPullingTime="18.216643695s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:24.102306987 +0000 UTC m=+27.836481220" watchObservedRunningTime="2026-09-07 10:20:24.103585166 +0000 UTC m=+27.837759398"
Sep 07 10:20:24.779020 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:24.778934    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:24.779584 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:24.779043    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:25.083506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:25.083410    2577 generic.go:356] "Generic (PLEG): container finished" podID="ac75d0ea-397b-457f-b8b1-46735f0dff26" containerID="2c1f0d7f9e94f1b0739a6d188c7b0d4d6d98c505166e584f0d2b6cea58056a5f" exitCode=0
Sep 07 10:20:25.083676 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:25.083495    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hp27c" event={"ID":"ac75d0ea-397b-457f-b8b1-46735f0dff26","Type":"ContainerDied","Data":"2c1f0d7f9e94f1b0739a6d188c7b0d4d6d98c505166e584f0d2b6cea58056a5f"}
Sep 07 10:20:25.778167 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:25.778107    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:25.778349 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:25.778117    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:25.778349 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:25.778292    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:25.778349 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:25.778325    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:25.814286 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:25.814255    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ktvqp"]
Sep 07 10:20:25.817839 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:25.817804    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x8w2s"]
Sep 07 10:20:25.817976 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:25.817913    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:25.818038 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:25.818009    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:25.830308 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:25.830259    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9xlk4"]
Sep 07 10:20:26.086199 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:26.086096    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:26.086199 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:26.086105    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:26.086556 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:26.086202    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:26.086663 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:26.086642    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:27.093501 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:27.093463    2577 generic.go:356] "Generic (PLEG): container finished" podID="ac75d0ea-397b-457f-b8b1-46735f0dff26" containerID="d9d132b715711798b72e36687cdbf15f11b24fa05c394b8540f96969b9023c4e" exitCode=0
Sep 07 10:20:27.093859 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:27.093542    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hp27c" event={"ID":"ac75d0ea-397b-457f-b8b1-46735f0dff26","Type":"ContainerDied","Data":"d9d132b715711798b72e36687cdbf15f11b24fa05c394b8540f96969b9023c4e"}
Sep 07 10:20:27.778244 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:27.778030    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:27.778398 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:27.778030    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:27.778398 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:27.778329    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:27.778398 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:27.778030    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:27.778398 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:27.778384    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:27.778565 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:27.778472    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:29.100194 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:29.100159    2577 generic.go:356] "Generic (PLEG): container finished" podID="ac75d0ea-397b-457f-b8b1-46735f0dff26" containerID="ae1c507f76bd15a1575de6b063bf8cb8848b312f779a0bdc35f3968e583529e3" exitCode=0
Sep 07 10:20:29.100563 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:29.100207    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hp27c" event={"ID":"ac75d0ea-397b-457f-b8b1-46735f0dff26","Type":"ContainerDied","Data":"ae1c507f76bd15a1575de6b063bf8cb8848b312f779a0bdc35f3968e583529e3"}
Sep 07 10:20:29.778787 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:29.778748    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:29.778787 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:29.778776    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:29.778987 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:29.778747    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:29.778987 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:29.778873    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x8w2s" podUID="415363e6-edcf-41e5-97ce-cd379cf30f4e"
Sep 07 10:20:29.778987 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:29.778955    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9xlk4" podUID="e537c01e-65d3-4e1f-ab15-84ddc4b0dc46"
Sep 07 10:20:29.779080 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:29.779040    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ktvqp" podUID="804db42a-4910-4e6f-bdd2-d62cbb720877"
Sep 07 10:20:30.446599 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:30.446561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs\") pod \"network-metrics-daemon-x8w2s\" (UID: \"415363e6-edcf-41e5-97ce-cd379cf30f4e\") " pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:30.447062 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:30.446655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:30.447062 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:30.446730    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 10:20:30.447062 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:30.446766    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 07 10:20:30.447062 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:30.446797    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs podName:415363e6-edcf-41e5-97ce-cd379cf30f4e nodeName:}" failed. No retries permitted until 2026-09-07 10:21:02.446781047 +0000 UTC m=+66.180955259 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs") pod "network-metrics-daemon-x8w2s" (UID: "415363e6-edcf-41e5-97ce-cd379cf30f4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 07 10:20:30.447062 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:30.446822    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret podName:804db42a-4910-4e6f-bdd2-d62cbb720877 nodeName:}" failed. No retries permitted until 2026-09-07 10:21:02.446804657 +0000 UTC m=+66.180978872 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret") pod "global-pull-secret-syncer-ktvqp" (UID: "804db42a-4910-4e6f-bdd2-d62cbb720877") : object "kube-system"/"original-pull-secret" not registered
Sep 07 10:20:30.648027 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:30.647990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jczdv\" (UniqueName: \"kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv\") pod \"network-check-target-9xlk4\" (UID: \"e537c01e-65d3-4e1f-ab15-84ddc4b0dc46\") " pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:30.648244 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:30.648198    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 07 10:20:30.648244 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:30.648224    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 07 10:20:30.648244 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:30.648237    2577 projected.go:196] Error preparing data for projected volume kube-api-access-jczdv for pod openshift-network-diagnostics/network-check-target-9xlk4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 10:20:30.648395 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:30.648305    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv podName:e537c01e-65d3-4e1f-ab15-84ddc4b0dc46 nodeName:}" failed. No retries permitted until 2026-09-07 10:21:02.648285183 +0000 UTC m=+66.382459415 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jczdv" (UniqueName: "kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv") pod "network-check-target-9xlk4" (UID: "e537c01e-65d3-4e1f-ab15-84ddc4b0dc46") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 07 10:20:31.067424 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.067394    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-166.ec2.internal" event="NodeReady"
Sep 07 10:20:31.067609 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.067531    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 07 10:20:31.100803 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.100771    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"]
Sep 07 10:20:31.117170 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.117118    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-x5vgs"]
Sep 07 10:20:31.117318 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.117257    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"
Sep 07 10:20:31.119961 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.119910    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 07 10:20:31.120197 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.120172    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 07 10:20:31.120287 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.120259    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 07 10:20:31.120428 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.120409    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-b5chh\""
Sep 07 10:20:31.137933 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.137895    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-87d987788-lpz4n"]
Sep 07 10:20:31.138099 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.138002    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-x5vgs"
Sep 07 10:20:31.142000 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.141795    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 07 10:20:31.142000 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.141860    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 07 10:20:31.145655 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.145346    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-v6xbh\""
Sep 07 10:20:31.158218 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.158184    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-srlkt"]
Sep 07 10:20:31.158370 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.158307    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.160829 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.160804    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 07 10:20:31.161124 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.161106    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-z7qs2\""
Sep 07 10:20:31.161238 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.161165    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 07 10:20:31.161398 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.161383    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 07 10:20:31.166470 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.166387    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 07 10:20:31.168232 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.168209    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"]
Sep 07 10:20:31.168367 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.168347    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.170796 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.170770    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 07 10:20:31.171156 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.170860    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-62qxv\""
Sep 07 10:20:31.171156 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.170872    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 07 10:20:31.171156 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.170874    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 07 10:20:31.171331 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.171202    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 07 10:20:31.176416 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.176395    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 07 10:20:31.180653 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.180633    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-grp6m"]
Sep 07 10:20:31.180725 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.180701    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:20:31.182838 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.182819    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 07 10:20:31.183019 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.183006    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 07 10:20:31.183096 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.183081    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 07 10:20:31.183323 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.183304    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pcfl4\""
Sep 07 10:20:31.184353 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.184340    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 07 10:20:31.196432 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.196403    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7676d8944-tzrbr"]
Sep 07 10:20:31.196561 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.196512    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:31.198898 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.198873    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 07 10:20:31.199024 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.198998    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 07 10:20:31.199024 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.199008    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 07 10:20:31.199198 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.199020    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-p7tkk\""
Sep 07 10:20:31.199198 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.199074    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 07 10:20:31.203829 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.203805    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"]
Sep 07 10:20:31.205291 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.203944    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.205291 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.204453    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn"]
Sep 07 10:20:31.205687 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.205668    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 07 10:20:31.208327 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.208308    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 07 10:20:31.208426 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.208370    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 07 10:20:31.208518 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.208502    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-lpsft\""
Sep 07 10:20:31.208640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.208569    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 07 10:20:31.208640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.208581    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 07 10:20:31.208755 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.208739    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 07 10:20:31.209026 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.209011    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 07 10:20:31.218514 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.218489    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-x5vgs"]
Sep 07 10:20:31.218627 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.218521    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-grp6m"]
Sep 07 10:20:31.218627 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.218537    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-nspbh"]
Sep 07 10:20:31.218627 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.218609    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn"
Sep 07 10:20:31.221225 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.221203    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 07 10:20:31.221332 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.221231    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 07 10:20:31.221332 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.221204    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 07 10:20:31.221541 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.221527    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-q9plb\""
Sep 07 10:20:31.221586 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.221554    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 07 10:20:31.225779 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.225759    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn"]
Sep 07 10:20:31.225904 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.225878    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-nspbh"
Sep 07 10:20:31.227964 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.227944    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 07 10:20:31.228387 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.228364    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 07 10:20:31.228486 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.228469    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xrftd\""
Sep 07 10:20:31.241492 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.241199    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tc4rv"]
Sep 07 10:20:31.241492 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.241340    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn"
Sep 07 10:20:31.244181 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.244152    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-m7zm8\""
Sep 07 10:20:31.244308 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.244257    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 07 10:20:31.244372 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.244359    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 07 10:20:31.244454 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.244438    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 07 10:20:31.244532 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.244516    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 07 10:20:31.250267 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.250243    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-87d987788-lpz4n"]
Sep 07 10:20:31.250357 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.250279    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5g6z4"]
Sep 07 10:20:31.250424 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.250404    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tc4rv"
Sep 07 10:20:31.252511 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.252492    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b38067b2-aa31-4846-b669-8588a67c8fdf-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-h87jj\" (UID: \"b38067b2-aa31-4846-b669-8588a67c8fdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:20:31.252631 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.252527    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-h87jj\" (UID: \"b38067b2-aa31-4846-b669-8588a67c8fdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:20:31.252631 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.252559    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/228ba212-8e2a-4dab-8381-cc40eb711df7-serving-cert\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.252631 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.252586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kx5d\" (UniqueName: \"kubernetes.io/projected/b2626e1c-b66e-4c2a-8d52-399b53bf16c8-kube-api-access-9kx5d\") pod \"volume-data-source-validator-695c895bd6-x5vgs\" (UID: \"b2626e1c-b66e-4c2a-8d52-399b53bf16c8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-x5vgs"
Sep 07 10:20:31.252777 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.252670    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/228ba212-8e2a-4dab-8381-cc40eb711df7-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.252777 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.252701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l4tc\" (UniqueName: \"kubernetes.io/projected/228ba212-8e2a-4dab-8381-cc40eb711df7-kube-api-access-7l4tc\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.252777 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.252731    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4rf5\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-kube-api-access-h4rf5\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.252919 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.252806    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2284375e-7edc-4b16-b3b6-336d9f25bda4-trusted-ca\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.252919 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.252850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/228ba212-8e2a-4dab-8381-cc40eb711df7-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.252919 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.252900    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-bound-sa-token\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.253057 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.252942    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/228ba212-8e2a-4dab-8381-cc40eb711df7-tmp\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.253057 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.252981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szbjd\" (UniqueName: \"kubernetes.io/projected/60d763d9-c798-44d1-a3b4-b00feb890428-kube-api-access-szbjd\") pod \"cluster-samples-operator-7cdcfcf76c-vngk6\" (UID: \"60d763d9-c798-44d1-a3b4-b00feb890428\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"
Sep 07 10:20:31.253057 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.253006    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.253228 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.253079    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2284375e-7edc-4b16-b3b6-336d9f25bda4-installation-pull-secrets\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.253228 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.253124    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/228ba212-8e2a-4dab-8381-cc40eb711df7-snapshots\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.253228 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.253187    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-vngk6\" (UID: \"60d763d9-c798-44d1-a3b4-b00feb890428\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"
Sep 07 10:20:31.253368 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.253295    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 07 10:20:31.253368 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.253262    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2284375e-7edc-4b16-b3b6-336d9f25bda4-image-registry-private-configuration\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.253368 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.253335    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-certificates\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.253507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.253382    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dppqh\" (UniqueName: \"kubernetes.io/projected/b38067b2-aa31-4846-b669-8588a67c8fdf-kube-api-access-dppqh\") pod \"cluster-monitoring-operator-6cdcc96965-h87jj\" (UID: \"b38067b2-aa31-4846-b669-8588a67c8fdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:20:31.253507 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.253405    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2284375e-7edc-4b16-b3b6-336d9f25bda4-ca-trust-extracted\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.253615 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.253514    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 07 10:20:31.253615 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.253530    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-594j9\""
Sep 07 10:20:31.253779 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.253752    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 07 10:20:31.264286 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.264255    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"]
Sep 07 10:20:31.264455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.264297    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-srlkt"]
Sep 07 10:20:31.264455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.264318    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7676d8944-tzrbr"]
Sep 07 10:20:31.264455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.264338    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn"]
Sep 07 10:20:31.264455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.264356    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-nspbh"]
Sep 07 10:20:31.264455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.264368    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:31.264455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.264381    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tc4rv"]
Sep 07 10:20:31.264455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.264400    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5g6z4"]
Sep 07 10:20:31.264455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.264423    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn"]
Sep 07 10:20:31.265735 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.265709    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:31.266656 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.266526    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vkvfh\""
Sep 07 10:20:31.266794 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.266774    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 07 10:20:31.266961 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.266946    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 07 10:20:31.353765 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.353672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-certificates\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.353765 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.353717    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dppqh\" (UniqueName: \"kubernetes.io/projected/b38067b2-aa31-4846-b669-8588a67c8fdf-kube-api-access-dppqh\") pod \"cluster-monitoring-operator-6cdcc96965-h87jj\" (UID: \"b38067b2-aa31-4846-b669-8588a67c8fdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:20:31.353765 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.353745    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/72995f6c-69b6-468a-86e2-2471b01e2e4d-config\") pod \"console-operator-85b454874b-grp6m\" (UID: \"72995f6c-69b6-468a-86e2-2471b01e2e4d\") " pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:31.354118 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354060    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-h87jj\" (UID: \"b38067b2-aa31-4846-b669-8588a67c8fdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:20:31.354118 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:31.354118 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-default-certificate\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.354118 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4rf5\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-kube-api-access-h4rf5\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.354615 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.354383    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 10:20:31.354615 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354394    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-certificates\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.354615 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354405    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ad9111f-27a6-4f1f-b886-66c18049ccb6-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8rktn\" (UID: \"1ad9111f-27a6-4f1f-b886-66c18049ccb6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn"
Sep 07 10:20:31.354615 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.354485    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls podName:b38067b2-aa31-4846-b669-8588a67c8fdf nodeName:}" failed. No retries permitted until 2026-09-07 10:20:31.854465593 +0000 UTC m=+35.588639823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-h87jj" (UID: "b38067b2-aa31-4846-b669-8588a67c8fdf") : secret "cluster-monitoring-operator-tls" not found
Sep 07 10:20:31.354615 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354556    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72995f6c-69b6-468a-86e2-2471b01e2e4d-trusted-ca\") pod \"console-operator-85b454874b-grp6m\" (UID: \"72995f6c-69b6-468a-86e2-2471b01e2e4d\") " pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:31.354615 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:31.354965 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354620    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/228ba212-8e2a-4dab-8381-cc40eb711df7-tmp\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.354965 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354648    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8crm\" (UniqueName: \"kubernetes.io/projected/1ad9111f-27a6-4f1f-b886-66c18049ccb6-kube-api-access-v8crm\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8rktn\" (UID: \"1ad9111f-27a6-4f1f-b886-66c18049ccb6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn"
Sep 07 10:20:31.354965 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354670    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.354965 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354688    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zql7s\" (UniqueName: \"kubernetes.io/projected/1fac04e7-2860-4151-88df-b67485cd4f19-kube-api-access-zql7s\") pod \"ingress-canary-tc4rv\" (UID: \"1fac04e7-2860-4151-88df-b67485cd4f19\") " pod="openshift-ingress-canary/ingress-canary-tc4rv"
Sep 07 10:20:31.354965 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b38067b2-aa31-4846-b669-8588a67c8fdf-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-h87jj\" (UID: \"b38067b2-aa31-4846-b669-8588a67c8fdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:20:31.354965 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354749    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/72995f6c-69b6-468a-86e2-2471b01e2e4d-serving-cert\") pod \"console-operator-85b454874b-grp6m\" (UID: \"72995f6c-69b6-468a-86e2-2471b01e2e4d\") " pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:31.354965 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354771    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce8a104b-d12d-46bf-accb-e8ed7d04195c-tmp-dir\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:31.354965 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2284375e-7edc-4b16-b3b6-336d9f25bda4-installation-pull-secrets\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.354965 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354855    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.354965 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354911    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/228ba212-8e2a-4dab-8381-cc40eb711df7-snapshots\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.354965 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354943    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-vngk6\" (UID: \"60d763d9-c798-44d1-a3b4-b00feb890428\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.354971    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2284375e-7edc-4b16-b3b6-336d9f25bda4-image-registry-private-configuration\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355000    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355007    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/228ba212-8e2a-4dab-8381-cc40eb711df7-tmp\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355026    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wllp\" (UniqueName: \"kubernetes.io/projected/bee903a3-a8e5-4120-9a8f-573d186cdf2c-kube-api-access-7wllp\") pod \"service-ca-operator-859f94c4b-zbhwn\" (UID: \"bee903a3-a8e5-4120-9a8f-573d186cdf2c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn"
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4shp\" (UniqueName: \"kubernetes.io/projected/72995f6c-69b6-468a-86e2-2471b01e2e4d-kube-api-access-n4shp\") pod \"console-operator-85b454874b-grp6m\" (UID: \"72995f6c-69b6-468a-86e2-2471b01e2e4d\") " pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355090    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bee903a3-a8e5-4120-9a8f-573d186cdf2c-serving-cert\") pod \"service-ca-operator-859f94c4b-zbhwn\" (UID: \"bee903a3-a8e5-4120-9a8f-573d186cdf2c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn"
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.355099    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2284375e-7edc-4b16-b3b6-336d9f25bda4-ca-trust-extracted\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.355182    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls podName:60d763d9-c798-44d1-a3b4-b00feb890428 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:31.855163376 +0000 UTC m=+35.589337601 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-vngk6" (UID: "60d763d9-c798-44d1-a3b4-b00feb890428") : secret "samples-operator-tls" not found
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355199    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce8a104b-d12d-46bf-accb-e8ed7d04195c-config-volume\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/228ba212-8e2a-4dab-8381-cc40eb711df7-serving-cert\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355255    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kx5d\" (UniqueName: \"kubernetes.io/projected/b2626e1c-b66e-4c2a-8d52-399b53bf16c8-kube-api-access-9kx5d\") pod \"volume-data-source-validator-695c895bd6-x5vgs\" (UID: \"b2626e1c-b66e-4c2a-8d52-399b53bf16c8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-x5vgs"
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/228ba212-8e2a-4dab-8381-cc40eb711df7-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355310    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l4tc\" (UniqueName: \"kubernetes.io/projected/228ba212-8e2a-4dab-8381-cc40eb711df7-kube-api-access-7l4tc\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355335    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1fac04e7-2860-4151-88df-b67485cd4f19-cert\") pod \"ingress-canary-tc4rv\" (UID: \"1fac04e7-2860-4151-88df-b67485cd4f19\") " pod="openshift-ingress-canary/ingress-canary-tc4rv"
Sep 07 10:20:31.355553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355362    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2284375e-7edc-4b16-b3b6-336d9f25bda4-trusted-ca\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355389    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv4h2\" (UniqueName: \"kubernetes.io/projected/612d739f-b714-4df4-8c5f-0a35c7c06c5b-kube-api-access-mv4h2\") pod \"network-check-source-685456655f-nspbh\" (UID: \"612d739f-b714-4df4-8c5f-0a35c7c06c5b\") " pod="openshift-network-diagnostics/network-check-source-685456655f-nspbh"
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/228ba212-8e2a-4dab-8381-cc40eb711df7-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355441    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2284375e-7edc-4b16-b3b6-336d9f25bda4-ca-trust-extracted\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggjwv\" (UniqueName: \"kubernetes.io/projected/945b76dd-21f5-4690-b0fb-0afbf104601d-kube-api-access-ggjwv\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355485    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v79hb\" (UniqueName: \"kubernetes.io/projected/ce8a104b-d12d-46bf-accb-e8ed7d04195c-kube-api-access-v79hb\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355492    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b38067b2-aa31-4846-b669-8588a67c8fdf-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-h87jj\" (UID: \"b38067b2-aa31-4846-b669-8588a67c8fdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-bound-sa-token\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szbjd\" (UniqueName: \"kubernetes.io/projected/60d763d9-c798-44d1-a3b4-b00feb890428-kube-api-access-szbjd\") pod \"cluster-samples-operator-7cdcfcf76c-vngk6\" (UID: \"60d763d9-c798-44d1-a3b4-b00feb890428\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ad9111f-27a6-4f1f-b886-66c18049ccb6-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8rktn\" (UID: \"1ad9111f-27a6-4f1f-b886-66c18049ccb6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn"
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355621    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bee903a3-a8e5-4120-9a8f-573d186cdf2c-config\") pod \"service-ca-operator-859f94c4b-zbhwn\" (UID: \"bee903a3-a8e5-4120-9a8f-573d186cdf2c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn"
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355652    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-stats-auth\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.355904    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/228ba212-8e2a-4dab-8381-cc40eb711df7-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.356008    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/228ba212-8e2a-4dab-8381-cc40eb711df7-snapshots\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.356313    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.356329    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87d987788-lpz4n: secret "image-registry-tls" not found
Sep 07 10:20:31.356493 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.356388    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls podName:2284375e-7edc-4b16-b3b6-336d9f25bda4 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:31.856370502 +0000 UTC m=+35.590544716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls") pod "image-registry-87d987788-lpz4n" (UID: "2284375e-7edc-4b16-b3b6-336d9f25bda4") : secret "image-registry-tls" not found
Sep 07 10:20:31.357111 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.356813    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/228ba212-8e2a-4dab-8381-cc40eb711df7-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.357111 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.356950    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2284375e-7edc-4b16-b3b6-336d9f25bda4-trusted-ca\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.360486 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.360465    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/228ba212-8e2a-4dab-8381-cc40eb711df7-serving-cert\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.360597 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.360517    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2284375e-7edc-4b16-b3b6-336d9f25bda4-image-registry-private-configuration\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.360597 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.360556    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2284375e-7edc-4b16-b3b6-336d9f25bda4-installation-pull-secrets\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.364684 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.364624    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szbjd\" (UniqueName: \"kubernetes.io/projected/60d763d9-c798-44d1-a3b4-b00feb890428-kube-api-access-szbjd\") pod \"cluster-samples-operator-7cdcfcf76c-vngk6\" (UID: \"60d763d9-c798-44d1-a3b4-b00feb890428\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"
Sep 07 10:20:31.365470 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.365420    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dppqh\" (UniqueName: \"kubernetes.io/projected/b38067b2-aa31-4846-b669-8588a67c8fdf-kube-api-access-dppqh\") pod \"cluster-monitoring-operator-6cdcc96965-h87jj\" (UID: \"b38067b2-aa31-4846-b669-8588a67c8fdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:20:31.365581 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.365501    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4rf5\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-kube-api-access-h4rf5\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.365635 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.365578    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-bound-sa-token\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.366621 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.366597    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l4tc\" (UniqueName: \"kubernetes.io/projected/228ba212-8e2a-4dab-8381-cc40eb711df7-kube-api-access-7l4tc\") pod \"insights-operator-76bcd6ccf5-srlkt\" (UID: \"228ba212-8e2a-4dab-8381-cc40eb711df7\") " pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.366952 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.366925    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kx5d\" (UniqueName: \"kubernetes.io/projected/b2626e1c-b66e-4c2a-8d52-399b53bf16c8-kube-api-access-9kx5d\") pod \"volume-data-source-validator-695c895bd6-x5vgs\" (UID: \"b2626e1c-b66e-4c2a-8d52-399b53bf16c8\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-x5vgs"
Sep 07 10:20:31.453764 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.453724    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-x5vgs"
Sep 07 10:20:31.456029 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.455993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8crm\" (UniqueName: \"kubernetes.io/projected/1ad9111f-27a6-4f1f-b886-66c18049ccb6-kube-api-access-v8crm\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8rktn\" (UID: \"1ad9111f-27a6-4f1f-b886-66c18049ccb6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn"
Sep 07 10:20:31.456199 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.456051    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zql7s\" (UniqueName: \"kubernetes.io/projected/1fac04e7-2860-4151-88df-b67485cd4f19-kube-api-access-zql7s\") pod \"ingress-canary-tc4rv\" (UID: \"1fac04e7-2860-4151-88df-b67485cd4f19\") " pod="openshift-ingress-canary/ingress-canary-tc4rv"
Sep 07 10:20:31.456199 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.456082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/72995f6c-69b6-468a-86e2-2471b01e2e4d-serving-cert\") pod \"console-operator-85b454874b-grp6m\" (UID: \"72995f6c-69b6-468a-86e2-2471b01e2e4d\") " pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:31.456199 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.456109    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce8a104b-d12d-46bf-accb-e8ed7d04195c-tmp-dir\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:31.456199 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.456143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.456199 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.456189    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.456488 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.456215    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wllp\" (UniqueName: \"kubernetes.io/projected/bee903a3-a8e5-4120-9a8f-573d186cdf2c-kube-api-access-7wllp\") pod \"service-ca-operator-859f94c4b-zbhwn\" (UID: \"bee903a3-a8e5-4120-9a8f-573d186cdf2c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn"
Sep 07 10:20:31.456488 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.456241    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4shp\" (UniqueName: \"kubernetes.io/projected/72995f6c-69b6-468a-86e2-2471b01e2e4d-kube-api-access-n4shp\") pod \"console-operator-85b454874b-grp6m\" (UID: \"72995f6c-69b6-468a-86e2-2471b01e2e4d\") " pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:31.456488 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.456277    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bee903a3-a8e5-4120-9a8f-573d186cdf2c-serving-cert\") pod \"service-ca-operator-859f94c4b-zbhwn\" (UID: \"bee903a3-a8e5-4120-9a8f-573d186cdf2c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn"
Sep 07 10:20:31.456488 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.456302    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce8a104b-d12d-46bf-accb-e8ed7d04195c-config-volume\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:31.456488 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.456335    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1fac04e7-2860-4151-88df-b67485cd4f19-cert\") pod \"ingress-canary-tc4rv\" (UID: \"1fac04e7-2860-4151-88df-b67485cd4f19\") " pod="openshift-ingress-canary/ingress-canary-tc4rv"
Sep 07 10:20:31.456488 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.456354    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 10:20:31.456488 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.456467    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs podName:945b76dd-21f5-4690-b0fb-0afbf104601d nodeName:}" failed. No retries permitted until 2026-09-07 10:20:31.956451396 +0000 UTC m=+35.690625607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs") pod "router-default-7676d8944-tzrbr" (UID: "945b76dd-21f5-4690-b0fb-0afbf104601d") : secret "router-metrics-certs-default" not found
Sep 07 10:20:31.456878 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.456656    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle podName:945b76dd-21f5-4690-b0fb-0afbf104601d nodeName:}" failed. No retries permitted until 2026-09-07 10:20:31.956638024 +0000 UTC m=+35.690812258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle") pod "router-default-7676d8944-tzrbr" (UID: "945b76dd-21f5-4690-b0fb-0afbf104601d") : configmap references non-existent config key: service-ca.crt
Sep 07 10:20:31.456878 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.456362    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv4h2\" (UniqueName: \"kubernetes.io/projected/612d739f-b714-4df4-8c5f-0a35c7c06c5b-kube-api-access-mv4h2\") pod \"network-check-source-685456655f-nspbh\" (UID: \"612d739f-b714-4df4-8c5f-0a35c7c06c5b\") " pod="openshift-network-diagnostics/network-check-source-685456655f-nspbh"
Sep 07 10:20:31.457037 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.456881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggjwv\" (UniqueName: \"kubernetes.io/projected/945b76dd-21f5-4690-b0fb-0afbf104601d-kube-api-access-ggjwv\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.457037 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.456925    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v79hb\" (UniqueName: \"kubernetes.io/projected/ce8a104b-d12d-46bf-accb-e8ed7d04195c-kube-api-access-v79hb\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:31.457037 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.457010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ad9111f-27a6-4f1f-b886-66c18049ccb6-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8rktn\" (UID: \"1ad9111f-27a6-4f1f-b886-66c18049ccb6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn"
Sep 07 10:20:31.457214 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.457045    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bee903a3-a8e5-4120-9a8f-573d186cdf2c-config\") pod \"service-ca-operator-859f94c4b-zbhwn\" (UID: \"bee903a3-a8e5-4120-9a8f-573d186cdf2c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn"
Sep 07 10:20:31.457214 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.457076    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-stats-auth\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.457214 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.457108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/72995f6c-69b6-468a-86e2-2471b01e2e4d-config\") pod \"console-operator-85b454874b-grp6m\" (UID: \"72995f6c-69b6-468a-86e2-2471b01e2e4d\") " pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:31.457214 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.457108    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce8a104b-d12d-46bf-accb-e8ed7d04195c-tmp-dir\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:31.457214 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.457175    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-default-certificate\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.457214 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.457210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ad9111f-27a6-4f1f-b886-66c18049ccb6-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8rktn\" (UID: \"1ad9111f-27a6-4f1f-b886-66c18049ccb6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn"
Sep 07 10:20:31.457748 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.457234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72995f6c-69b6-468a-86e2-2471b01e2e4d-trusted-ca\") pod \"console-operator-85b454874b-grp6m\" (UID: \"72995f6c-69b6-468a-86e2-2471b01e2e4d\") " pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:31.457748 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.457262    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:31.457748 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.457390    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 10:20:31.457748 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.457448    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls podName:ce8a104b-d12d-46bf-accb-e8ed7d04195c nodeName:}" failed. No retries permitted until 2026-09-07 10:20:31.95743307 +0000 UTC m=+35.691607286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls") pod "dns-default-5g6z4" (UID: "ce8a104b-d12d-46bf-accb-e8ed7d04195c") : secret "dns-default-metrics-tls" not found
Sep 07 10:20:31.457748 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.457582    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce8a104b-d12d-46bf-accb-e8ed7d04195c-config-volume\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:31.458422 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.458341    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/72995f6c-69b6-468a-86e2-2471b01e2e4d-config\") pod \"console-operator-85b454874b-grp6m\" (UID: \"72995f6c-69b6-468a-86e2-2471b01e2e4d\") " pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:31.459005 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.458978    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/72995f6c-69b6-468a-86e2-2471b01e2e4d-trusted-ca\") pod \"console-operator-85b454874b-grp6m\" (UID: \"72995f6c-69b6-468a-86e2-2471b01e2e4d\") " pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:31.459092 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.459051    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ad9111f-27a6-4f1f-b886-66c18049ccb6-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8rktn\" (UID: \"1ad9111f-27a6-4f1f-b886-66c18049ccb6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn"
Sep 07 10:20:31.459749 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.459678    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bee903a3-a8e5-4120-9a8f-573d186cdf2c-serving-cert\") pod \"service-ca-operator-859f94c4b-zbhwn\" (UID: \"bee903a3-a8e5-4120-9a8f-573d186cdf2c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn"
Sep 07 10:20:31.459953 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.459930    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bee903a3-a8e5-4120-9a8f-573d186cdf2c-config\") pod \"service-ca-operator-859f94c4b-zbhwn\" (UID: \"bee903a3-a8e5-4120-9a8f-573d186cdf2c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn"
Sep 07 10:20:31.460356 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.460331    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-stats-auth\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.460445 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.460360    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/72995f6c-69b6-468a-86e2-2471b01e2e4d-serving-cert\") pod \"console-operator-85b454874b-grp6m\" (UID: \"72995f6c-69b6-468a-86e2-2471b01e2e4d\") " pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:31.460656 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.460639    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1fac04e7-2860-4151-88df-b67485cd4f19-cert\") pod \"ingress-canary-tc4rv\" (UID: \"1fac04e7-2860-4151-88df-b67485cd4f19\") " pod="openshift-ingress-canary/ingress-canary-tc4rv"
Sep 07 10:20:31.462435 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.462411    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ad9111f-27a6-4f1f-b886-66c18049ccb6-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8rktn\" (UID: \"1ad9111f-27a6-4f1f-b886-66c18049ccb6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn"
Sep 07 10:20:31.462523 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.462434    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-default-certificate\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.468558 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.468480    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggjwv\" (UniqueName: \"kubernetes.io/projected/945b76dd-21f5-4690-b0fb-0afbf104601d-kube-api-access-ggjwv\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.469346 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.469318    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8crm\" (UniqueName: \"kubernetes.io/projected/1ad9111f-27a6-4f1f-b886-66c18049ccb6-kube-api-access-v8crm\") pod \"kube-storage-version-migrator-operator-8d7dd4675-8rktn\" (UID: \"1ad9111f-27a6-4f1f-b886-66c18049ccb6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn"
Sep 07 10:20:31.469508 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.469471    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wllp\" (UniqueName: \"kubernetes.io/projected/bee903a3-a8e5-4120-9a8f-573d186cdf2c-kube-api-access-7wllp\") pod \"service-ca-operator-859f94c4b-zbhwn\" (UID: \"bee903a3-a8e5-4120-9a8f-573d186cdf2c\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn"
Sep 07 10:20:31.469937 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.469908    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv4h2\" (UniqueName: \"kubernetes.io/projected/612d739f-b714-4df4-8c5f-0a35c7c06c5b-kube-api-access-mv4h2\") pod \"network-check-source-685456655f-nspbh\" (UID: \"612d739f-b714-4df4-8c5f-0a35c7c06c5b\") " pod="openshift-network-diagnostics/network-check-source-685456655f-nspbh"
Sep 07 10:20:31.470402 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.470379    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4shp\" (UniqueName: \"kubernetes.io/projected/72995f6c-69b6-468a-86e2-2471b01e2e4d-kube-api-access-n4shp\") pod \"console-operator-85b454874b-grp6m\" (UID: \"72995f6c-69b6-468a-86e2-2471b01e2e4d\") " pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:31.470859 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.470839    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zql7s\" (UniqueName: \"kubernetes.io/projected/1fac04e7-2860-4151-88df-b67485cd4f19-kube-api-access-zql7s\") pod \"ingress-canary-tc4rv\" (UID: \"1fac04e7-2860-4151-88df-b67485cd4f19\") " pod="openshift-ingress-canary/ingress-canary-tc4rv"
Sep 07 10:20:31.471406 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.471386    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v79hb\" (UniqueName: \"kubernetes.io/projected/ce8a104b-d12d-46bf-accb-e8ed7d04195c-kube-api-access-v79hb\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:31.478911 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.478886    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt"
Sep 07 10:20:31.506194 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.506169    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:31.528193 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.528166    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn"
Sep 07 10:20:31.536199 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.536163    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-nspbh"
Sep 07 10:20:31.552785 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.552754    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn"
Sep 07 10:20:31.565817 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.565569    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tc4rv"
Sep 07 10:20:31.768946 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.768914    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-x5vgs"]
Sep 07 10:20:31.771576 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.771549    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-srlkt"]
Sep 07 10:20:31.773388 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:20:31.773364    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2626e1c_b66e_4c2a_8d52_399b53bf16c8.slice/crio-bbe356cda3be6c3041cb4b20637f9b86d796d44f12fb204ec0189dfcb0dbd2bc WatchSource:0}: Error finding container bbe356cda3be6c3041cb4b20637f9b86d796d44f12fb204ec0189dfcb0dbd2bc: Status 404 returned error can't find the container with id bbe356cda3be6c3041cb4b20637f9b86d796d44f12fb204ec0189dfcb0dbd2bc
Sep 07 10:20:31.774861 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:20:31.774836    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod228ba212_8e2a_4dab_8381_cc40eb711df7.slice/crio-973005b740469872038743382135ed8e90bbae94e6b29df36a86f6db5ddb9212 WatchSource:0}: Error finding container 973005b740469872038743382135ed8e90bbae94e6b29df36a86f6db5ddb9212: Status 404 returned error can't find the container with id 973005b740469872038743382135ed8e90bbae94e6b29df36a86f6db5ddb9212
Sep 07 10:20:31.778759 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.778655    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:20:31.778759 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.778699    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:20:31.778759 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.778699    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:20:31.781301 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.781277    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 10:20:31.781449 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.781320    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 10:20:31.781449 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.781361    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lbx74\""
Sep 07 10:20:31.781449 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.781286    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9jrgm\""
Sep 07 10:20:31.824109 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.824078    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-grp6m"]
Sep 07 10:20:31.828488 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:20:31.828454    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72995f6c_69b6_468a_86e2_2471b01e2e4d.slice/crio-0f92ba90877c90d70b1725dfdc6a07ddcfeb7b1e243d6f28a8f4a792735d985a WatchSource:0}: Error finding container 0f92ba90877c90d70b1725dfdc6a07ddcfeb7b1e243d6f28a8f4a792735d985a: Status 404 returned error can't find the container with id 0f92ba90877c90d70b1725dfdc6a07ddcfeb7b1e243d6f28a8f4a792735d985a
Sep 07 10:20:31.855602 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.855493    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn"]
Sep 07 10:20:31.858166 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:20:31.858117    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ad9111f_27a6_4f1f_b886_66c18049ccb6.slice/crio-707131a39056f3d1fd3618f49dc9beb93ba50048948f23cc5cc3f7b6bf227bc1 WatchSource:0}: Error finding container 707131a39056f3d1fd3618f49dc9beb93ba50048948f23cc5cc3f7b6bf227bc1: Status 404 returned error can't find the container with id 707131a39056f3d1fd3618f49dc9beb93ba50048948f23cc5cc3f7b6bf227bc1
Sep 07 10:20:31.861837 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.861803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:31.861941 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.861862    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-vngk6\" (UID: \"60d763d9-c798-44d1-a3b4-b00feb890428\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"
Sep 07 10:20:31.861941 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.861916    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 10:20:31.861941 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.861933    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87d987788-lpz4n: secret "image-registry-tls" not found
Sep 07 10:20:31.862084 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.861985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-h87jj\" (UID: \"b38067b2-aa31-4846-b669-8588a67c8fdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:20:31.862084 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.862002    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls podName:2284375e-7edc-4b16-b3b6-336d9f25bda4 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:32.861983426 +0000 UTC m=+36.596157651 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls") pod "image-registry-87d987788-lpz4n" (UID: "2284375e-7edc-4b16-b3b6-336d9f25bda4") : secret "image-registry-tls" not found
Sep 07 10:20:31.862084 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.862028    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 10:20:31.862084 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.862063    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 10:20:31.862329 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.862087    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls podName:60d763d9-c798-44d1-a3b4-b00feb890428 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:32.862068494 +0000 UTC m=+36.596242718 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-vngk6" (UID: "60d763d9-c798-44d1-a3b4-b00feb890428") : secret "samples-operator-tls" not found
Sep 07 10:20:31.862329 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.862110    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls podName:b38067b2-aa31-4846-b669-8588a67c8fdf nodeName:}" failed. No retries permitted until 2026-09-07 10:20:32.862095409 +0000 UTC m=+36.596269647 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-h87jj" (UID: "b38067b2-aa31-4846-b669-8588a67c8fdf") : secret "cluster-monitoring-operator-tls" not found
Sep 07 10:20:31.872835 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.872766    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-nspbh"]
Sep 07 10:20:31.875842 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:20:31.875820    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod612d739f_b714_4df4_8c5f_0a35c7c06c5b.slice/crio-721c71a3df0fef006a94df87166ea9468daaa6ba163c26f8a6e3e541e33161c3 WatchSource:0}: Error finding container 721c71a3df0fef006a94df87166ea9468daaa6ba163c26f8a6e3e541e33161c3: Status 404 returned error can't find the container with id 721c71a3df0fef006a94df87166ea9468daaa6ba163c26f8a6e3e541e33161c3
Sep 07 10:20:31.896381 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.896281    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn"]
Sep 07 10:20:31.901047 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:20:31.901020    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbee903a3_a8e5_4120_9a8f_573d186cdf2c.slice/crio-46a9cc1e3c99a48848210c535806b5a62437f9a42390d2d08a41eaa5755a44bc WatchSource:0}: Error finding container 46a9cc1e3c99a48848210c535806b5a62437f9a42390d2d08a41eaa5755a44bc: Status 404 returned error can't find the container with id 46a9cc1e3c99a48848210c535806b5a62437f9a42390d2d08a41eaa5755a44bc
Sep 07 10:20:31.914605 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.914571    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tc4rv"]
Sep 07 10:20:31.919613 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:20:31.919583    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fac04e7_2860_4151_88df_b67485cd4f19.slice/crio-ef31e8a15fd8742c1152d83bba9f0881d1403f7af00fcb807c73fbe6741fee8c WatchSource:0}: Error finding container ef31e8a15fd8742c1152d83bba9f0881d1403f7af00fcb807c73fbe6741fee8c: Status 404 returned error can't find the container with id ef31e8a15fd8742c1152d83bba9f0881d1403f7af00fcb807c73fbe6741fee8c
Sep 07 10:20:31.963015 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.962976    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:31.963235 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.963035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.963235 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:31.963076    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:31.963235 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.963126    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 10:20:31.963235 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.963211    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls podName:ce8a104b-d12d-46bf-accb-e8ed7d04195c nodeName:}" failed. No retries permitted until 2026-09-07 10:20:32.963195637 +0000 UTC m=+36.697369848 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls") pod "dns-default-5g6z4" (UID: "ce8a104b-d12d-46bf-accb-e8ed7d04195c") : secret "dns-default-metrics-tls" not found
Sep 07 10:20:31.963235 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.963211    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 10:20:31.963235 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.963232    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle podName:945b76dd-21f5-4690-b0fb-0afbf104601d nodeName:}" failed. No retries permitted until 2026-09-07 10:20:32.963216827 +0000 UTC m=+36.697391055 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle") pod "router-default-7676d8944-tzrbr" (UID: "945b76dd-21f5-4690-b0fb-0afbf104601d") : configmap references non-existent config key: service-ca.crt
Sep 07 10:20:31.963494 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:31.963256    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs podName:945b76dd-21f5-4690-b0fb-0afbf104601d nodeName:}" failed. No retries permitted until 2026-09-07 10:20:32.96324582 +0000 UTC m=+36.697420034 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs") pod "router-default-7676d8944-tzrbr" (UID: "945b76dd-21f5-4690-b0fb-0afbf104601d") : secret "router-metrics-certs-default" not found
Sep 07 10:20:32.110166 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:32.110109    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tc4rv" event={"ID":"1fac04e7-2860-4151-88df-b67485cd4f19","Type":"ContainerStarted","Data":"ef31e8a15fd8742c1152d83bba9f0881d1403f7af00fcb807c73fbe6741fee8c"}
Sep 07 10:20:32.111383 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:32.111356    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt" event={"ID":"228ba212-8e2a-4dab-8381-cc40eb711df7","Type":"ContainerStarted","Data":"973005b740469872038743382135ed8e90bbae94e6b29df36a86f6db5ddb9212"}
Sep 07 10:20:32.112728 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:32.112677    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn" event={"ID":"1ad9111f-27a6-4f1f-b886-66c18049ccb6","Type":"ContainerStarted","Data":"707131a39056f3d1fd3618f49dc9beb93ba50048948f23cc5cc3f7b6bf227bc1"}
Sep 07 10:20:32.113861 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:32.113834    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-grp6m" event={"ID":"72995f6c-69b6-468a-86e2-2471b01e2e4d","Type":"ContainerStarted","Data":"0f92ba90877c90d70b1725dfdc6a07ddcfeb7b1e243d6f28a8f4a792735d985a"}
Sep 07 10:20:32.115077 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:32.115058    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn" event={"ID":"bee903a3-a8e5-4120-9a8f-573d186cdf2c","Type":"ContainerStarted","Data":"46a9cc1e3c99a48848210c535806b5a62437f9a42390d2d08a41eaa5755a44bc"}
Sep 07 10:20:32.116177 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:32.116158    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-nspbh" event={"ID":"612d739f-b714-4df4-8c5f-0a35c7c06c5b","Type":"ContainerStarted","Data":"721c71a3df0fef006a94df87166ea9468daaa6ba163c26f8a6e3e541e33161c3"}
Sep 07 10:20:32.117192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:32.117167    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-x5vgs" event={"ID":"b2626e1c-b66e-4c2a-8d52-399b53bf16c8","Type":"ContainerStarted","Data":"bbe356cda3be6c3041cb4b20637f9b86d796d44f12fb204ec0189dfcb0dbd2bc"}
Sep 07 10:20:32.871855 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:32.871813    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:32.872600 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:32.871893    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-vngk6\" (UID: \"60d763d9-c798-44d1-a3b4-b00feb890428\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"
Sep 07 10:20:32.872600 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:32.872010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-h87jj\" (UID: \"b38067b2-aa31-4846-b669-8588a67c8fdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:20:32.872600 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:32.872191    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 10:20:32.872600 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:32.872256    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls podName:b38067b2-aa31-4846-b669-8588a67c8fdf nodeName:}" failed. No retries permitted until 2026-09-07 10:20:34.872236825 +0000 UTC m=+38.606411061 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-h87jj" (UID: "b38067b2-aa31-4846-b669-8588a67c8fdf") : secret "cluster-monitoring-operator-tls" not found
Sep 07 10:20:32.872600 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:32.872331    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 10:20:32.872600 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:32.872343    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87d987788-lpz4n: secret "image-registry-tls" not found
Sep 07 10:20:32.872600 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:32.872381    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls podName:2284375e-7edc-4b16-b3b6-336d9f25bda4 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:34.8723697 +0000 UTC m=+38.606543926 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls") pod "image-registry-87d987788-lpz4n" (UID: "2284375e-7edc-4b16-b3b6-336d9f25bda4") : secret "image-registry-tls" not found
Sep 07 10:20:32.872600 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:32.872435    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 10:20:32.872600 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:32.872468    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls podName:60d763d9-c798-44d1-a3b4-b00feb890428 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:34.872457182 +0000 UTC m=+38.606631407 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-vngk6" (UID: "60d763d9-c798-44d1-a3b4-b00feb890428") : secret "samples-operator-tls" not found
Sep 07 10:20:32.973220 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:32.973163    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:32.973410 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:32.973242    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:32.973472 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:32.973450    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:32.973648 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:32.973597    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 10:20:32.973724 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:32.973664    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls podName:ce8a104b-d12d-46bf-accb-e8ed7d04195c nodeName:}" failed. No retries permitted until 2026-09-07 10:20:34.973643596 +0000 UTC m=+38.707817826 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls") pod "dns-default-5g6z4" (UID: "ce8a104b-d12d-46bf-accb-e8ed7d04195c") : secret "dns-default-metrics-tls" not found
Sep 07 10:20:32.974096 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:32.974075    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 10:20:32.974203 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:32.974153    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs podName:945b76dd-21f5-4690-b0fb-0afbf104601d nodeName:}" failed. No retries permitted until 2026-09-07 10:20:34.974119062 +0000 UTC m=+38.708293291 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs") pod "router-default-7676d8944-tzrbr" (UID: "945b76dd-21f5-4690-b0fb-0afbf104601d") : secret "router-metrics-certs-default" not found
Sep 07 10:20:32.974292 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:32.974277    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle podName:945b76dd-21f5-4690-b0fb-0afbf104601d nodeName:}" failed. No retries permitted until 2026-09-07 10:20:34.974263701 +0000 UTC m=+38.708437921 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle") pod "router-default-7676d8944-tzrbr" (UID: "945b76dd-21f5-4690-b0fb-0afbf104601d") : configmap references non-existent config key: service-ca.crt
Sep 07 10:20:33.268934 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:33.268848    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:33.340770 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:33.340588    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:33.430717 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:33.430600    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:33.524936 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:33.524803    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:33.627436 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:33.627210    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:33.711316 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:33.711231    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:33.814575 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:33.814461    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:33.912739 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:33.912673    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:34.585480 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:34.585274    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:34.643431 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:34.643394    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:34.892391 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:34.892315    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:34.892391 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:34.892381    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-vngk6\" (UID: \"60d763d9-c798-44d1-a3b4-b00feb890428\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"
Sep 07 10:20:34.892559 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:34.892480    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 10:20:34.892559 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:34.892498    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87d987788-lpz4n: secret "image-registry-tls" not found
Sep 07 10:20:34.892559 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:34.892500    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-h87jj\" (UID: \"b38067b2-aa31-4846-b669-8588a67c8fdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:20:34.892671 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:34.892554    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 10:20:34.892671 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:34.892571    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls podName:2284375e-7edc-4b16-b3b6-336d9f25bda4 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:38.892551388 +0000 UTC m=+42.626725615 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls") pod "image-registry-87d987788-lpz4n" (UID: "2284375e-7edc-4b16-b3b6-336d9f25bda4") : secret "image-registry-tls" not found
Sep 07 10:20:34.892671 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:34.892604    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 10:20:34.892671 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:34.892622    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls podName:60d763d9-c798-44d1-a3b4-b00feb890428 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:38.892604017 +0000 UTC m=+42.626778245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-vngk6" (UID: "60d763d9-c798-44d1-a3b4-b00feb890428") : secret "samples-operator-tls" not found
Sep 07 10:20:34.892671 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:34.892642    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls podName:b38067b2-aa31-4846-b669-8588a67c8fdf nodeName:}" failed. No retries permitted until 2026-09-07 10:20:38.89263126 +0000 UTC m=+42.626805472 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-h87jj" (UID: "b38067b2-aa31-4846-b669-8588a67c8fdf") : secret "cluster-monitoring-operator-tls" not found
Sep 07 10:20:34.992907 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:34.992867    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:34.993561 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:34.993354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:34.993561 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:34.993413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:34.993561 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:34.993461    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:34.993561 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:34.993538    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 10:20:34.993809 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:34.993608    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls podName:ce8a104b-d12d-46bf-accb-e8ed7d04195c nodeName:}" failed. No retries permitted until 2026-09-07 10:20:38.993588223 +0000 UTC m=+42.727762456 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls") pod "dns-default-5g6z4" (UID: "ce8a104b-d12d-46bf-accb-e8ed7d04195c") : secret "dns-default-metrics-tls" not found
Sep 07 10:20:34.993809 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:34.993635    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle podName:945b76dd-21f5-4690-b0fb-0afbf104601d nodeName:}" failed. No retries permitted until 2026-09-07 10:20:38.993619715 +0000 UTC m=+42.727793946 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle") pod "router-default-7676d8944-tzrbr" (UID: "945b76dd-21f5-4690-b0fb-0afbf104601d") : configmap references non-existent config key: service-ca.crt
Sep 07 10:20:34.993809 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:34.993700    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 10:20:34.993809 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:34.993736    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs podName:945b76dd-21f5-4690-b0fb-0afbf104601d nodeName:}" failed. No retries permitted until 2026-09-07 10:20:38.9937256 +0000 UTC m=+42.727899837 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs") pod "router-default-7676d8944-tzrbr" (UID: "945b76dd-21f5-4690-b0fb-0afbf104601d") : secret "router-metrics-certs-default" not found
Sep 07 10:20:35.051004 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:35.050965    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:35.789871 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:35.789763    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:36.178086 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:36.177984    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:36.281467 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:36.281411    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:37.911269 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:37.911227    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:38.011867 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:38.011831    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:38.933049 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:38.933006    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:38.933553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:38.933072    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-vngk6\" (UID: \"60d763d9-c798-44d1-a3b4-b00feb890428\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"
Sep 07 10:20:38.933553 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:38.933191    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 10:20:38.933553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:38.933202    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-h87jj\" (UID: \"b38067b2-aa31-4846-b669-8588a67c8fdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:20:38.933553 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:38.933217    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87d987788-lpz4n: secret "image-registry-tls" not found
Sep 07 10:20:38.933553 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:38.933283    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls podName:2284375e-7edc-4b16-b3b6-336d9f25bda4 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:46.933259546 +0000 UTC m=+50.667433770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls") pod "image-registry-87d987788-lpz4n" (UID: "2284375e-7edc-4b16-b3b6-336d9f25bda4") : secret "image-registry-tls" not found
Sep 07 10:20:38.933553 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:38.933284    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 10:20:38.933553 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:38.933292    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 10:20:38.933553 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:38.933347    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls podName:60d763d9-c798-44d1-a3b4-b00feb890428 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:46.93332839 +0000 UTC m=+50.667502601 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-vngk6" (UID: "60d763d9-c798-44d1-a3b4-b00feb890428") : secret "samples-operator-tls" not found
Sep 07 10:20:38.933553 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:38.933370    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls podName:b38067b2-aa31-4846-b669-8588a67c8fdf nodeName:}" failed. No retries permitted until 2026-09-07 10:20:46.933358491 +0000 UTC m=+50.667532702 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-h87jj" (UID: "b38067b2-aa31-4846-b669-8588a67c8fdf") : secret "cluster-monitoring-operator-tls" not found
Sep 07 10:20:39.034337 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:39.034302    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:39.034532 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:39.034364    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:39.034532 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:39.034415    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:39.034532 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:39.034475    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 10:20:39.034532 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:39.034478    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 10:20:39.034705 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:39.034545    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle podName:945b76dd-21f5-4690-b0fb-0afbf104601d nodeName:}" failed. No retries permitted until 2026-09-07 10:20:47.034529047 +0000 UTC m=+50.768703258 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle") pod "router-default-7676d8944-tzrbr" (UID: "945b76dd-21f5-4690-b0fb-0afbf104601d") : configmap references non-existent config key: service-ca.crt
Sep 07 10:20:39.034705 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:39.034565    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls podName:ce8a104b-d12d-46bf-accb-e8ed7d04195c nodeName:}" failed. No retries permitted until 2026-09-07 10:20:47.034558165 +0000 UTC m=+50.768732376 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls") pod "dns-default-5g6z4" (UID: "ce8a104b-d12d-46bf-accb-e8ed7d04195c") : secret "dns-default-metrics-tls" not found
Sep 07 10:20:39.034705 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:39.034581    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs podName:945b76dd-21f5-4690-b0fb-0afbf104601d nodeName:}" failed. No retries permitted until 2026-09-07 10:20:47.034573152 +0000 UTC m=+50.768747363 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs") pod "router-default-7676d8944-tzrbr" (UID: "945b76dd-21f5-4690-b0fb-0afbf104601d") : secret "router-metrics-certs-default" not found
Sep 07 10:20:42.157150 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:42.156970    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tc4rv" event={"ID":"1fac04e7-2860-4151-88df-b67485cd4f19","Type":"ContainerStarted","Data":"2720f4a4748bcccb50789597c84b66354ceb331b0a64c750220342a360b339ca"}
Sep 07 10:20:42.168094 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:42.167956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn" event={"ID":"1ad9111f-27a6-4f1f-b886-66c18049ccb6","Type":"ContainerStarted","Data":"ef0ed5fcd1af90a7eb2f00e5df6d3be9408eedc3d15d1581864d2fd4b974c5d5"}
Sep 07 10:20:42.170553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:42.170527    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/0.log"
Sep 07 10:20:42.170757 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:42.170738    2577 generic.go:356] "Generic (PLEG): container finished" podID="72995f6c-69b6-468a-86e2-2471b01e2e4d" containerID="194b9770682de4514c5fbe2ffcbe6371d30406840a15413900e919c7d7bb8781" exitCode=255
Sep 07 10:20:42.170968 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:42.170872    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-grp6m" event={"ID":"72995f6c-69b6-468a-86e2-2471b01e2e4d","Type":"ContainerDied","Data":"194b9770682de4514c5fbe2ffcbe6371d30406840a15413900e919c7d7bb8781"}
Sep 07 10:20:42.171072 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:42.170998    2577 scope.go:117] "RemoveContainer" containerID="194b9770682de4514c5fbe2ffcbe6371d30406840a15413900e919c7d7bb8781"
Sep 07 10:20:42.173885 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:42.173614    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn" event={"ID":"bee903a3-a8e5-4120-9a8f-573d186cdf2c","Type":"ContainerStarted","Data":"0be39ffce509388c1a4eee12b8aa78188bb0fe5e07475bfba4038f5b705af940"}
Sep 07 10:20:42.177024 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:42.176980    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tc4rv" podStartSLOduration=1.2284487689999999 podStartE2EDuration="11.176954738s" totalImagesPullingTime="9.948505969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:20:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:42.174966969 +0000 UTC m=+45.909141203" watchObservedRunningTime="2026-09-07 10:20:42.176954738 +0000 UTC m=+45.911128968"
Sep 07 10:20:42.177689 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:42.177660    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-nspbh" event={"ID":"612d739f-b714-4df4-8c5f-0a35c7c06c5b","Type":"ContainerStarted","Data":"e714bc5af68751e28564e638b26411d08881bfe7a9160e0a95993f6035f6a2d3"}
Sep 07 10:20:42.186113 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:42.186084    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-x5vgs" event={"ID":"b2626e1c-b66e-4c2a-8d52-399b53bf16c8","Type":"ContainerStarted","Data":"456a7dab19ff3b3396663d5ccf6a9848dc01e6d7289df6b898367074e6786729"}
Sep 07 10:20:42.217669 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:42.217617    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn" podStartSLOduration=19.204811954 podStartE2EDuration="29.216045476s" totalImagesPullingTime="10.011233522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:20:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:42.19464001 +0000 UTC m=+45.928814244" watchObservedRunningTime="2026-09-07 10:20:42.216045476 +0000 UTC m=+45.950219712"
Sep 07 10:20:42.233369 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:42.233309    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn" podStartSLOduration=16.265833623 podStartE2EDuration="26.233270094s" totalImagesPullingTime="9.967436471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:42.233036546 +0000 UTC m=+45.967210795" watchObservedRunningTime="2026-09-07 10:20:42.233270094 +0000 UTC m=+45.967444328"
Sep 07 10:20:42.251301 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:42.251226    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-x5vgs" podStartSLOduration=24.15556775 podStartE2EDuration="34.251204299s" totalImagesPullingTime="10.095636549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:42.250438499 +0000 UTC m=+45.984612732" watchObservedRunningTime="2026-09-07 10:20:42.251204299 +0000 UTC m=+45.985378537"
Sep 07 10:20:42.329719 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:42.329622    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:42.757941 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:42.757911    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:43.192101 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.192064    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt" event={"ID":"228ba212-8e2a-4dab-8381-cc40eb711df7","Type":"ContainerStarted","Data":"a0ce011bb9c59ff614df2375abae6b88b8367888d87c99eadaa586e3197e71f7"}
Sep 07 10:20:43.196066 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.196035    2577 generic.go:356] "Generic (PLEG): container finished" podID="ac75d0ea-397b-457f-b8b1-46735f0dff26" containerID="bfc47c7c90f0663fbd0ec106b1a2baef2316ef98df1a7dfe635c7c568d250fb3" exitCode=0
Sep 07 10:20:43.196243 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.196120    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hp27c" event={"ID":"ac75d0ea-397b-457f-b8b1-46735f0dff26","Type":"ContainerDied","Data":"bfc47c7c90f0663fbd0ec106b1a2baef2316ef98df1a7dfe635c7c568d250fb3"}
Sep 07 10:20:43.200113 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.200092    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:20:43.202002 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.200596    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/0.log"
Sep 07 10:20:43.202002 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.200626    2577 generic.go:356] "Generic (PLEG): container finished" podID="72995f6c-69b6-468a-86e2-2471b01e2e4d" containerID="96add42daa5d6cb16c77f6a534f76533778ed728309193ccc1dd0912c558de0c" exitCode=255
Sep 07 10:20:43.202002 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.201424    2577 scope.go:117] "RemoveContainer" containerID="96add42daa5d6cb16c77f6a534f76533778ed728309193ccc1dd0912c558de0c"
Sep 07 10:20:43.202002 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:43.201562    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-grp6m_openshift-console-operator(72995f6c-69b6-468a-86e2-2471b01e2e4d)\"" pod="openshift-console-operator/console-operator-85b454874b-grp6m" podUID="72995f6c-69b6-468a-86e2-2471b01e2e4d"
Sep 07 10:20:43.202002 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.201597    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-grp6m" event={"ID":"72995f6c-69b6-468a-86e2-2471b01e2e4d","Type":"ContainerDied","Data":"96add42daa5d6cb16c77f6a534f76533778ed728309193ccc1dd0912c558de0c"}
Sep 07 10:20:43.202002 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.201632    2577 scope.go:117] "RemoveContainer" containerID="194b9770682de4514c5fbe2ffcbe6371d30406840a15413900e919c7d7bb8781"
Sep 07 10:20:43.235077 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.235020    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt" podStartSLOduration=25.134669151 podStartE2EDuration="35.234997133s" totalImagesPullingTime="10.100327982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:43.21643139 +0000 UTC m=+46.950605624" watchObservedRunningTime="2026-09-07 10:20:43.234997133 +0000 UTC m=+46.969171367"
Sep 07 10:20:43.281041 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.280957    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-nspbh" podStartSLOduration=17.287317259 podStartE2EDuration="27.280933095s" totalImagesPullingTime="9.993615836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:43.252107624 +0000 UTC m=+46.986281858" watchObservedRunningTime="2026-09-07 10:20:43.280933095 +0000 UTC m=+47.015107331"
Sep 07 10:20:43.704080 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.704040    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-djsc2"]
Sep 07 10:20:43.721491 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.721458    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-djsc2"]
Sep 07 10:20:43.721664 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.721593    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2"
Sep 07 10:20:43.724188 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.724013    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 07 10:20:43.724188 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.724096    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8ppc5\""
Sep 07 10:20:43.724530 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.724511    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 07 10:20:43.829898 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.829852    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:43.886070 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.886031    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-nginx-conf\") pod \"networking-console-plugin-79977f9844-djsc2\" (UID: \"dc5e172d-81d8-4dd8-9ff7-26010c0ef971\") " pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2"
Sep 07 10:20:43.886292 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.886239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-djsc2\" (UID: \"dc5e172d-81d8-4dd8-9ff7-26010c0ef971\") " pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2"
Sep 07 10:20:43.916058 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.916023    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-p9h4t"]
Sep 07 10:20:43.916521 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.916474    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:43.931757 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.931715    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p9h4t"]
Sep 07 10:20:43.931953 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.931853    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:43.935315 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.934796    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xz5mc\""
Sep 07 10:20:43.935315 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.935045    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 07 10:20:43.935315 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.935057    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 07 10:20:43.978791 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.978714    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:43.986913 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.986885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-djsc2\" (UID: \"dc5e172d-81d8-4dd8-9ff7-26010c0ef971\") " pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2"
Sep 07 10:20:43.987078 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.986961    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-nginx-conf\") pod \"networking-console-plugin-79977f9844-djsc2\" (UID: \"dc5e172d-81d8-4dd8-9ff7-26010c0ef971\") " pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2"
Sep 07 10:20:43.987078 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:43.987051    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 10:20:43.987210 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:43.987168    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert podName:dc5e172d-81d8-4dd8-9ff7-26010c0ef971 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:44.487119535 +0000 UTC m=+48.221293763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-djsc2" (UID: "dc5e172d-81d8-4dd8-9ff7-26010c0ef971") : secret "networking-console-plugin-cert" not found
Sep 07 10:20:43.987631 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:43.987613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-nginx-conf\") pod \"networking-console-plugin-79977f9844-djsc2\" (UID: \"dc5e172d-81d8-4dd8-9ff7-26010c0ef971\") " pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2"
Sep 07 10:20:44.075389 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.075358    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:44.087913 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.087878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-data-volume\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:44.088106 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.087930    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:44.088106 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.087963    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx947\" (UniqueName: \"kubernetes.io/projected/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-kube-api-access-jx947\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:44.088106 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.088095    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:44.088310 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.088184    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-crio-socket\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:44.122533 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.122501    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-lsg9w"]
Sep 07 10:20:44.139323 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.139284    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-lsg9w"]
Sep 07 10:20:44.139498 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.139399    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-lsg9w"
Sep 07 10:20:44.141755 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.141718    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 07 10:20:44.141913 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.141895    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-wq6ln\""
Sep 07 10:20:44.141966 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.141954    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 07 10:20:44.165332 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.165300    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:44.189606 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.189562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-data-volume\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:44.189606 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.189608    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:44.189786 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.189627    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx947\" (UniqueName: \"kubernetes.io/projected/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-kube-api-access-jx947\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:44.189786 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:44.189734    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 10:20:44.189860 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:44.189799    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls podName:1189ab74-04bb-4104-ae57-cc5fe35a3fbd nodeName:}" failed. No retries permitted until 2026-09-07 10:20:44.689783713 +0000 UTC m=+48.423957940 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9h4t" (UID: "1189ab74-04bb-4104-ae57-cc5fe35a3fbd") : secret "insights-runtime-extractor-tls" not found
Sep 07 10:20:44.189860 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.189825    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:44.189959 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.189889    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-crio-socket\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:44.189959 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:44.189892    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 10:20:44.189959 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.189923    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-data-volume\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:44.189959 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:44.189955    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-kube-rbac-proxy-cm podName:1189ab74-04bb-4104-ae57-cc5fe35a3fbd nodeName:}" failed. No retries permitted until 2026-09-07 10:20:44.689937751 +0000 UTC m=+48.424111976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9h4t" (UID: "1189ab74-04bb-4104-ae57-cc5fe35a3fbd") : configmap "kube-rbac-proxy" not found
Sep 07 10:20:44.190154 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.190085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-crio-socket\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:44.205029 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.204998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx947\" (UniqueName: \"kubernetes.io/projected/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-kube-api-access-jx947\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:44.208267 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.208237    2577 generic.go:356] "Generic (PLEG): container finished" podID="ac75d0ea-397b-457f-b8b1-46735f0dff26" containerID="1b55857ce3e4823f5b0d0ed969ab9414a9e2ef60cba27332d9b5e97156d95a00" exitCode=0
Sep 07 10:20:44.208405 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.208317    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hp27c" event={"ID":"ac75d0ea-397b-457f-b8b1-46735f0dff26","Type":"ContainerDied","Data":"1b55857ce3e4823f5b0d0ed969ab9414a9e2ef60cba27332d9b5e97156d95a00"}
Sep 07 10:20:44.210263 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.210244    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:20:44.210709 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.210693    2577 scope.go:117] "RemoveContainer" containerID="96add42daa5d6cb16c77f6a534f76533778ed728309193ccc1dd0912c558de0c"
Sep 07 10:20:44.210928 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:44.210852    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-grp6m_openshift-console-operator(72995f6c-69b6-468a-86e2-2471b01e2e4d)\"" pod="openshift-console-operator/console-operator-85b454874b-grp6m" podUID="72995f6c-69b6-468a-86e2-2471b01e2e4d"
Sep 07 10:20:44.223634 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.223602    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:44.291245 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.291218    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-889tq\" (UniqueName: \"kubernetes.io/projected/7766f9d3-1516-4284-8a1b-fb443dc30fa8-kube-api-access-889tq\") pod \"migrator-fdbb8fdff-lsg9w\" (UID: \"7766f9d3-1516-4284-8a1b-fb443dc30fa8\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-lsg9w"
Sep 07 10:20:44.392266 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.392205    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-889tq\" (UniqueName: \"kubernetes.io/projected/7766f9d3-1516-4284-8a1b-fb443dc30fa8-kube-api-access-889tq\") pod \"migrator-fdbb8fdff-lsg9w\" (UID: \"7766f9d3-1516-4284-8a1b-fb443dc30fa8\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-lsg9w"
Sep 07 10:20:44.400436 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.400407    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-889tq\" (UniqueName: \"kubernetes.io/projected/7766f9d3-1516-4284-8a1b-fb443dc30fa8-kube-api-access-889tq\") pod \"migrator-fdbb8fdff-lsg9w\" (UID: \"7766f9d3-1516-4284-8a1b-fb443dc30fa8\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-lsg9w"
Sep 07 10:20:44.449731 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.449696    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-lsg9w"
Sep 07 10:20:44.494054 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.493966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-djsc2\" (UID: \"dc5e172d-81d8-4dd8-9ff7-26010c0ef971\") " pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2"
Sep 07 10:20:44.494226 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:44.494087    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 10:20:44.494226 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:44.494181    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert podName:dc5e172d-81d8-4dd8-9ff7-26010c0ef971 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:45.494162128 +0000 UTC m=+49.228336343 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-djsc2" (UID: "dc5e172d-81d8-4dd8-9ff7-26010c0ef971") : secret "networking-console-plugin-cert" not found
Sep 07 10:20:44.495148 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.495107    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:44.596409 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.596376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:44.696403 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.696359    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:44.696600 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.696464    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:44.696600 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:44.696530    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 10:20:44.696732 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:44.696607    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 10:20:44.696732 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:44.696616    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls podName:1189ab74-04bb-4104-ae57-cc5fe35a3fbd nodeName:}" failed. No retries permitted until 2026-09-07 10:20:45.696593844 +0000 UTC m=+49.430768076 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9h4t" (UID: "1189ab74-04bb-4104-ae57-cc5fe35a3fbd") : secret "insights-runtime-extractor-tls" not found
Sep 07 10:20:44.696732 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:44.696670    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-kube-rbac-proxy-cm podName:1189ab74-04bb-4104-ae57-cc5fe35a3fbd nodeName:}" failed. No retries permitted until 2026-09-07 10:20:45.696651646 +0000 UTC m=+49.430825857 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9h4t" (UID: "1189ab74-04bb-4104-ae57-cc5fe35a3fbd") : configmap "kube-rbac-proxy" not found
Sep 07 10:20:44.736260 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.736228    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-lsg9w"]
Sep 07 10:20:44.770547 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.770507    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:44.874235 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.874191    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:44.874681 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:44.874667    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4hg64_0187e151-799f-402c-9886-6cb9b0bba10d/dns-node-resolver/0.log"
Sep 07 10:20:45.217606 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:45.217570    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hp27c" event={"ID":"ac75d0ea-397b-457f-b8b1-46735f0dff26","Type":"ContainerStarted","Data":"d88a91441322c0ad2828d5e0ae84a6bcceebee4c999fd040d0324af3ed179b2f"}
Sep 07 10:20:45.218772 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:45.218754    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-lsg9w" event={"ID":"7766f9d3-1516-4284-8a1b-fb443dc30fa8","Type":"ContainerStarted","Data":"c1582d0007254855e1b859b41961de35f15f9ba1384056eed70532480d6f13d7"}
Sep 07 10:20:45.241556 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:45.241504    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hp27c" podStartSLOduration=10.060740272 podStartE2EDuration="48.241485299s" totalImagesPullingTime="38.180745027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:19:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:45.239292434 +0000 UTC m=+48.973466667" watchObservedRunningTime="2026-09-07 10:20:45.241485299 +0000 UTC m=+48.975659533"
Sep 07 10:20:45.359589 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:45.359556    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:45.406432 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:45.406318    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:45.504474 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:45.504374    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-djsc2\" (UID: \"dc5e172d-81d8-4dd8-9ff7-26010c0ef971\") " pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2"
Sep 07 10:20:45.504620 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:45.504537    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 10:20:45.504620 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:45.504616    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert podName:dc5e172d-81d8-4dd8-9ff7-26010c0ef971 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:47.504593164 +0000 UTC m=+51.238767377 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-djsc2" (UID: "dc5e172d-81d8-4dd8-9ff7-26010c0ef971") : secret "networking-console-plugin-cert" not found
Sep 07 10:20:45.705865 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:45.705824    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:45.706042 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:45.705945    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:45.706042 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:45.706001    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 10:20:45.706255 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:45.706088    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls podName:1189ab74-04bb-4104-ae57-cc5fe35a3fbd nodeName:}" failed. No retries permitted until 2026-09-07 10:20:47.706065691 +0000 UTC m=+51.440239908 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9h4t" (UID: "1189ab74-04bb-4104-ae57-cc5fe35a3fbd") : secret "insights-runtime-extractor-tls" not found
Sep 07 10:20:45.706255 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:45.706089    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 07 10:20:45.706255 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:45.706162    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-kube-rbac-proxy-cm podName:1189ab74-04bb-4104-ae57-cc5fe35a3fbd nodeName:}" failed. No retries permitted until 2026-09-07 10:20:47.706146567 +0000 UTC m=+51.440320790 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9h4t" (UID: "1189ab74-04bb-4104-ae57-cc5fe35a3fbd") : configmap "kube-rbac-proxy" not found
Sep 07 10:20:46.073235 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:46.073206    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mljbz_bcc4fa7a-3956-4a04-8fc7-b1914d6079f8/node-ca/0.log"
Sep 07 10:20:46.223668 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:46.223632    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-lsg9w" event={"ID":"7766f9d3-1516-4284-8a1b-fb443dc30fa8","Type":"ContainerStarted","Data":"1a95a4add3eef62b7835cb8bcae8641487c852d5b0566372a470b22beed570d1"}
Sep 07 10:20:46.224053 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:46.223693    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-lsg9w" event={"ID":"7766f9d3-1516-4284-8a1b-fb443dc30fa8","Type":"ContainerStarted","Data":"b83038e68f7becbb379b7af8459069af03f5065323dcc1bb050d469b195753d1"}
Sep 07 10:20:46.239931 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:46.239877    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-lsg9w" podStartSLOduration=1.037787409 podStartE2EDuration="2.239860152s" totalImagesPullingTime="1.202072743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:20:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:20:46.238649374 +0000 UTC m=+49.972823598" watchObservedRunningTime="2026-09-07 10:20:46.239860152 +0000 UTC m=+49.974034385"
Sep 07 10:20:46.330058 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:46.330025    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:46.407538 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:46.407504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:46.461901 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:46.461864    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:46.673508 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:46.673431    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tc4rv_1fac04e7-2860-4151-88df-b67485cd4f19/serve-healthcheck-canary/0.log"
Sep 07 10:20:47.018354 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:47.018316    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-h87jj\" (UID: \"b38067b2-aa31-4846-b669-8588a67c8fdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:20:47.018504 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:47.018396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:20:47.018504 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:47.018448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-vngk6\" (UID: \"60d763d9-c798-44d1-a3b4-b00feb890428\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"
Sep 07 10:20:47.018504 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.018481    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 07 10:20:47.018611 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.018554    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls podName:b38067b2-aa31-4846-b669-8588a67c8fdf nodeName:}" failed. No retries permitted until 2026-09-07 10:21:03.018532562 +0000 UTC m=+66.752706789 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-h87jj" (UID: "b38067b2-aa31-4846-b669-8588a67c8fdf") : secret "cluster-monitoring-operator-tls" not found
Sep 07 10:20:47.018611 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.018566    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 07 10:20:47.018611 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.018568    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 07 10:20:47.018611 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.018589    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87d987788-lpz4n: secret "image-registry-tls" not found
Sep 07 10:20:47.018611 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.018598    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls podName:60d763d9-c798-44d1-a3b4-b00feb890428 nodeName:}" failed. No retries permitted until 2026-09-07 10:21:03.018587922 +0000 UTC m=+66.752762142 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-vngk6" (UID: "60d763d9-c798-44d1-a3b4-b00feb890428") : secret "samples-operator-tls" not found
Sep 07 10:20:47.018769 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.018652    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls podName:2284375e-7edc-4b16-b3b6-336d9f25bda4 nodeName:}" failed. No retries permitted until 2026-09-07 10:21:03.018637139 +0000 UTC m=+66.752811353 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls") pod "image-registry-87d987788-lpz4n" (UID: "2284375e-7edc-4b16-b3b6-336d9f25bda4") : secret "image-registry-tls" not found
Sep 07 10:20:47.119509 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:47.119478    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:20:47.119622 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:47.119529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:47.119654 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.119627    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 07 10:20:47.119654 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.119627    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 07 10:20:47.119713 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:47.119663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:20:47.119713 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.119694    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls podName:ce8a104b-d12d-46bf-accb-e8ed7d04195c nodeName:}" failed. No retries permitted until 2026-09-07 10:21:03.119678578 +0000 UTC m=+66.853852789 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls") pod "dns-default-5g6z4" (UID: "ce8a104b-d12d-46bf-accb-e8ed7d04195c") : secret "dns-default-metrics-tls" not found
Sep 07 10:20:47.119776 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.119758    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs podName:945b76dd-21f5-4690-b0fb-0afbf104601d nodeName:}" failed. No retries permitted until 2026-09-07 10:21:03.119731626 +0000 UTC m=+66.853905845 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs") pod "router-default-7676d8944-tzrbr" (UID: "945b76dd-21f5-4690-b0fb-0afbf104601d") : secret "router-metrics-certs-default" not found
Sep 07 10:20:47.119812 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.119797    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle podName:945b76dd-21f5-4690-b0fb-0afbf104601d nodeName:}" failed. No retries permitted until 2026-09-07 10:21:03.119785828 +0000 UTC m=+66.853960042 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle") pod "router-default-7676d8944-tzrbr" (UID: "945b76dd-21f5-4690-b0fb-0afbf104601d") : configmap references non-existent config key: service-ca.crt
Sep 07 10:20:47.276310 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:47.276217    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:47.522177 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:47.522118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-djsc2\" (UID: \"dc5e172d-81d8-4dd8-9ff7-26010c0ef971\") " pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2"
Sep 07 10:20:47.522372 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.522287    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 10:20:47.522372 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.522366    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert podName:dc5e172d-81d8-4dd8-9ff7-26010c0ef971 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:51.522345788 +0000 UTC m=+55.256519999 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-djsc2" (UID: "dc5e172d-81d8-4dd8-9ff7-26010c0ef971") : secret "networking-console-plugin-cert" not found
Sep 07 10:20:47.719238 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:47.719202    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:47.723903 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:47.723881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:47.724029 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:47.723975    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:47.724029 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.724015    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 10:20:47.724110 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:47.724084    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls podName:1189ab74-04bb-4104-ae57-cc5fe35a3fbd nodeName:}" failed. No retries permitted until 2026-09-07 10:20:51.724066625 +0000 UTC m=+55.458240852 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9h4t" (UID: "1189ab74-04bb-4104-ae57-cc5fe35a3fbd") : secret "insights-runtime-extractor-tls" not found
Sep 07 10:20:47.724555 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:47.724537    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:49.527956 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:49.527917    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:50.476755 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:50.476708    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:50.527828 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:50.527794    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:50.719416 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:50.719384    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:50.822429 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:50.822339    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:51.507114 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:51.507060    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:51.507317 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:51.507161    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:20:51.507513 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:51.507501    2577 scope.go:117] "RemoveContainer" containerID="96add42daa5d6cb16c77f6a534f76533778ed728309193ccc1dd0912c558de0c"
Sep 07 10:20:51.507668 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:51.507651    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-grp6m_openshift-console-operator(72995f6c-69b6-468a-86e2-2471b01e2e4d)\"" pod="openshift-console-operator/console-operator-85b454874b-grp6m" podUID="72995f6c-69b6-468a-86e2-2471b01e2e4d"
Sep 07 10:20:51.554881 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:51.554840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-djsc2\" (UID: \"dc5e172d-81d8-4dd8-9ff7-26010c0ef971\") " pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2"
Sep 07 10:20:51.555051 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:51.555005    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 10:20:51.555090 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:51.555081    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert podName:dc5e172d-81d8-4dd8-9ff7-26010c0ef971 nodeName:}" failed. No retries permitted until 2026-09-07 10:20:59.555060751 +0000 UTC m=+63.289234974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-djsc2" (UID: "dc5e172d-81d8-4dd8-9ff7-26010c0ef971") : secret "networking-console-plugin-cert" not found
Sep 07 10:20:51.757033 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:51.756992    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:51.757507 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:51.757116    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 07 10:20:51.757507 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:51.757187    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls podName:1189ab74-04bb-4104-ae57-cc5fe35a3fbd nodeName:}" failed. No retries permitted until 2026-09-07 10:20:59.757173036 +0000 UTC m=+63.491347247 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9h4t" (UID: "1189ab74-04bb-4104-ae57-cc5fe35a3fbd") : secret "insights-runtime-extractor-tls" not found
Sep 07 10:20:51.963619 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:51.963578    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:52.063945 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:52.063853    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:52.164632 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:52.164586    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:52.267062 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:52.267033    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:52.367771 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:52.367686    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:52.470893 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:52.470859    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:52.663016 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:52.662926    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:52.913696 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:52.913590    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:52.970344 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:52.970300    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:53.667866 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:53.667834    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:53.767489 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:53.767450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:54.060974 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:54.060922    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:54.269415 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:54.269376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:54.461506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:54.461476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:54.860918 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:54.860827    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:55.462794 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:55.462762    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:55.658252 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:55.658216    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:55.880747 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:55.880656    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:56.062141 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:56.062101    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:56.103679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:56.103644    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-m4b2f"
Sep 07 10:20:56.261881 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:56.261842    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:56.683848 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:56.683777    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:56.783438 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:56.783406    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:56.884079 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:56.884039    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:57.066992 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:57.066956    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:57.759375 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:57.759339    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:58.056608 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:58.056513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:58.457521 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:58.457491    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:58.669758 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:58.669719    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:58.863360 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:58.863270    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:59.063338 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:59.063305    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:20:59.630426 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:59.630384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-djsc2\" (UID: \"dc5e172d-81d8-4dd8-9ff7-26010c0ef971\") " pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2"
Sep 07 10:20:59.630609 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:59.630540    2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 07 10:20:59.630651 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:20:59.630612    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert podName:dc5e172d-81d8-4dd8-9ff7-26010c0ef971 nodeName:}" failed. No retries permitted until 2026-09-07 10:21:15.630595457 +0000 UTC m=+79.364769686 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-djsc2" (UID: "dc5e172d-81d8-4dd8-9ff7-26010c0ef971") : secret "networking-console-plugin-cert" not found
Sep 07 10:20:59.831530 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:59.831498    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:59.834080 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:59.834055    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1189ab74-04bb-4104-ae57-cc5fe35a3fbd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9h4t\" (UID: \"1189ab74-04bb-4104-ae57-cc5fe35a3fbd\") " pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:20:59.846514 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:59.846493    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xz5mc\""
Sep 07 10:20:59.855382 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:20:59.855362    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p9h4t"
Sep 07 10:21:00.123249 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:00.123086    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p9h4t"]
Sep 07 10:21:00.125713 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:21:00.125680    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1189ab74_04bb_4104_ae57_cc5fe35a3fbd.slice/crio-3398356271ae22b319137b45c090a532ff72e8a3d214699a62ecf9f338c89759 WatchSource:0}: Error finding container 3398356271ae22b319137b45c090a532ff72e8a3d214699a62ecf9f338c89759: Status 404 returned error can't find the container with id 3398356271ae22b319137b45c090a532ff72e8a3d214699a62ecf9f338c89759
Sep 07 10:21:00.264771 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:00.264735    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:00.273720 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:00.273685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9h4t" event={"ID":"1189ab74-04bb-4104-ae57-cc5fe35a3fbd","Type":"ContainerStarted","Data":"be12935502c899393aaf1439fef37b7ae6bbf1010e0a02bf61d53c23a85ec43c"}
Sep 07 10:21:00.273873 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:00.273743    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9h4t" event={"ID":"1189ab74-04bb-4104-ae57-cc5fe35a3fbd","Type":"ContainerStarted","Data":"3398356271ae22b319137b45c090a532ff72e8a3d214699a62ecf9f338c89759"}
Sep 07 10:21:00.458406 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:00.458370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:00.660147 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:00.660098    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:00.863638 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:00.863547    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:01.057947 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:01.057913    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:01.281834 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:01.281794    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9h4t" event={"ID":"1189ab74-04bb-4104-ae57-cc5fe35a3fbd","Type":"ContainerStarted","Data":"f6f1b861bba7959d2bffc0f2fbd03ae806ae1ca87c480bbc18e1d63a41214b52"}
Sep 07 10:21:02.062077 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.062041    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:02.276908 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.276823    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:02.457970 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.457932    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:21:02.458484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.457988    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs\") pod \"network-metrics-daemon-x8w2s\" (UID: \"415363e6-edcf-41e5-97ce-cd379cf30f4e\") " pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:21:02.460478 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.460454    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 07 10:21:02.461433 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.461412    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 07 10:21:02.467954 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.467924    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:02.471324 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.471295    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/415363e6-edcf-41e5-97ce-cd379cf30f4e-metrics-certs\") pod \"network-metrics-daemon-x8w2s\" (UID: \"415363e6-edcf-41e5-97ce-cd379cf30f4e\") " pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:21:02.471444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.471350    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/804db42a-4910-4e6f-bdd2-d62cbb720877-original-pull-secret\") pod \"global-pull-secret-syncer-ktvqp\" (UID: \"804db42a-4910-4e6f-bdd2-d62cbb720877\") " pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:21:02.660316 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.660284    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jczdv\" (UniqueName: \"kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv\") pod \"network-check-target-9xlk4\" (UID: \"e537c01e-65d3-4e1f-ab15-84ddc4b0dc46\") " pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:21:02.662533 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.662499    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:02.662994 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.662975    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jczdv\" (UniqueName: \"kubernetes.io/projected/e537c01e-65d3-4e1f-ab15-84ddc4b0dc46-kube-api-access-jczdv\") pod \"network-check-target-9xlk4\" (UID: \"e537c01e-65d3-4e1f-ab15-84ddc4b0dc46\") " pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:21:02.693660 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.693633    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lbx74\""
Sep 07 10:21:02.700893 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.700867    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9jrgm\""
Sep 07 10:21:02.701091 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.701076    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x8w2s"
Sep 07 10:21:02.706236 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.705054    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ktvqp"
Sep 07 10:21:02.708785 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.708740    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:21:02.974845 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.974773    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ktvqp"]
Sep 07 10:21:02.978195 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:21:02.978163    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod804db42a_4910_4e6f_bdd2_d62cbb720877.slice/crio-3b617b7309fce1a7910c6a4ead070d85ee1baa71835d4ba589782adef56feb51 WatchSource:0}: Error finding container 3b617b7309fce1a7910c6a4ead070d85ee1baa71835d4ba589782adef56feb51: Status 404 returned error can't find the container with id 3b617b7309fce1a7910c6a4ead070d85ee1baa71835d4ba589782adef56feb51
Sep 07 10:21:02.988629 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.988541    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x8w2s"]
Sep 07 10:21:02.991211 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:02.991188    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9xlk4"]
Sep 07 10:21:02.991404 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:21:02.991381    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod415363e6_edcf_41e5_97ce_cd379cf30f4e.slice/crio-740d7293b9573860fbfd5e310f56422103ad2a87579fbc881db9c09376809d00 WatchSource:0}: Error finding container 740d7293b9573860fbfd5e310f56422103ad2a87579fbc881db9c09376809d00: Status 404 returned error can't find the container with id 740d7293b9573860fbfd5e310f56422103ad2a87579fbc881db9c09376809d00
Sep 07 10:21:02.993950 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:21:02.993911    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode537c01e_65d3_4e1f_ab15_84ddc4b0dc46.slice/crio-18d629519f8df95442a3f0a729dee1e57a4856bc9063178fa953e07f580936c6 WatchSource:0}: Error finding container 18d629519f8df95442a3f0a729dee1e57a4856bc9063178fa953e07f580936c6: Status 404 returned error can't find the container with id 18d629519f8df95442a3f0a729dee1e57a4856bc9063178fa953e07f580936c6
Sep 07 10:21:03.064108 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.064082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-h87jj\" (UID: \"b38067b2-aa31-4846-b669-8588a67c8fdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:21:03.064220 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.064161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:21:03.064220 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.064204    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-vngk6\" (UID: \"60d763d9-c798-44d1-a3b4-b00feb890428\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"
Sep 07 10:21:03.066631 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.066609    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/60d763d9-c798-44d1-a3b4-b00feb890428-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-vngk6\" (UID: \"60d763d9-c798-44d1-a3b4-b00feb890428\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"
Sep 07 10:21:03.067060 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.067042    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls\") pod \"image-registry-87d987788-lpz4n\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") " pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:21:03.067203 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.067185    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b38067b2-aa31-4846-b669-8588a67c8fdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-h87jj\" (UID: \"b38067b2-aa31-4846-b669-8588a67c8fdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:21:03.165468 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.165430    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:21:03.165468 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.165472    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:21:03.165722 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.165531    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:21:03.166171 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.166113    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/945b76dd-21f5-4690-b0fb-0afbf104601d-service-ca-bundle\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:21:03.167929 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.167907    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce8a104b-d12d-46bf-accb-e8ed7d04195c-metrics-tls\") pod \"dns-default-5g6z4\" (UID: \"ce8a104b-d12d-46bf-accb-e8ed7d04195c\") " pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:21:03.168018 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.167970    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/945b76dd-21f5-4690-b0fb-0afbf104601d-metrics-certs\") pod \"router-default-7676d8944-tzrbr\" (UID: \"945b76dd-21f5-4690-b0fb-0afbf104601d\") " pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:21:03.235430 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.235340    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-b5chh\""
Sep 07 10:21:03.243221 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.243198    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"
Sep 07 10:21:03.271654 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.271626    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-z7qs2\""
Sep 07 10:21:03.279764 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.279729    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:21:03.290125 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.290094    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ktvqp" event={"ID":"804db42a-4910-4e6f-bdd2-d62cbb720877","Type":"ContainerStarted","Data":"3b617b7309fce1a7910c6a4ead070d85ee1baa71835d4ba589782adef56feb51"}
Sep 07 10:21:03.290329 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.290304    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pcfl4\""
Sep 07 10:21:03.292867 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.292837    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9h4t" event={"ID":"1189ab74-04bb-4104-ae57-cc5fe35a3fbd","Type":"ContainerStarted","Data":"9abb39879c17ccadf903238f89847354e44364f3bd61183b37a6afbe588001ae"}
Sep 07 10:21:03.298367 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.298347    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"
Sep 07 10:21:03.298533 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.298506    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9xlk4" event={"ID":"e537c01e-65d3-4e1f-ab15-84ddc4b0dc46","Type":"ContainerStarted","Data":"9c5aaccbde97c0c13c3acd64884a683e83e1789e1e0ac8c2c3260b6afa152902"}
Sep 07 10:21:03.298613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.298559    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9xlk4" event={"ID":"e537c01e-65d3-4e1f-ab15-84ddc4b0dc46","Type":"ContainerStarted","Data":"18d629519f8df95442a3f0a729dee1e57a4856bc9063178fa953e07f580936c6"}
Sep 07 10:21:03.298674 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.298624    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:21:03.300956 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.300929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x8w2s" event={"ID":"415363e6-edcf-41e5-97ce-cd379cf30f4e","Type":"ContainerStarted","Data":"740d7293b9573860fbfd5e310f56422103ad2a87579fbc881db9c09376809d00"}
Sep 07 10:21:03.310099 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.310050    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-p9h4t" podStartSLOduration=17.98538133 podStartE2EDuration="20.310028536s" totalImagesPullingTime="2.324647206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:20:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:03.308924561 +0000 UTC m=+67.043098795" watchObservedRunningTime="2026-09-07 10:21:03.310028536 +0000 UTC m=+67.044202766"
Sep 07 10:21:03.315714 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.315686    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-lpsft\""
Sep 07 10:21:03.322037 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.321994    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9xlk4" podStartSLOduration=66.321979054 podStartE2EDuration="1m6.321979054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:03.321723227 +0000 UTC m=+67.055897471" watchObservedRunningTime="2026-09-07 10:21:03.321979054 +0000 UTC m=+67.056153287"
Sep 07 10:21:03.324295 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.324227    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:21:03.377569 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.377505    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vkvfh\""
Sep 07 10:21:03.386020 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.385807    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:21:03.517407 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.517333    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6"]
Sep 07 10:21:03.564270 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.564245    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-87d987788-lpz4n"]
Sep 07 10:21:03.566049 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:21:03.566023    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2284375e_7edc_4b16_b3b6_336d9f25bda4.slice/crio-8e5b6cd255cd8d4241c30d5b65425a331298f513d308451d7a1c9eda5a841399 WatchSource:0}: Error finding container 8e5b6cd255cd8d4241c30d5b65425a331298f513d308451d7a1c9eda5a841399: Status 404 returned error can't find the container with id 8e5b6cd255cd8d4241c30d5b65425a331298f513d308451d7a1c9eda5a841399
Sep 07 10:21:03.587213 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.587121    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj"]
Sep 07 10:21:03.592716 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:21:03.592686    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb38067b2_aa31_4846_b669_8588a67c8fdf.slice/crio-41b714336c026b06ed3708ca6497c6786eeff1bfb65cba118be738892d636d0f WatchSource:0}: Error finding container 41b714336c026b06ed3708ca6497c6786eeff1bfb65cba118be738892d636d0f: Status 404 returned error can't find the container with id 41b714336c026b06ed3708ca6497c6786eeff1bfb65cba118be738892d636d0f
Sep 07 10:21:03.611247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.611222    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7676d8944-tzrbr"]
Sep 07 10:21:03.613329 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:21:03.613301    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945b76dd_21f5_4690_b0fb_0afbf104601d.slice/crio-e654f3a442ef4cf75541f013c62e9782307313c5475d382ad5d737c43de8aa11 WatchSource:0}: Error finding container e654f3a442ef4cf75541f013c62e9782307313c5475d382ad5d737c43de8aa11: Status 404 returned error can't find the container with id e654f3a442ef4cf75541f013c62e9782307313c5475d382ad5d737c43de8aa11
Sep 07 10:21:03.659060 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:03.658992    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5g6z4"]
Sep 07 10:21:03.664341 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:21:03.664284    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce8a104b_d12d_46bf_accb_e8ed7d04195c.slice/crio-96e994af8c3a9ab4461fddbc07604c48823c2589d0bc9c1d2211c9cedbc6ced6 WatchSource:0}: Error finding container 96e994af8c3a9ab4461fddbc07604c48823c2589d0bc9c1d2211c9cedbc6ced6: Status 404 returned error can't find the container with id 96e994af8c3a9ab4461fddbc07604c48823c2589d0bc9c1d2211c9cedbc6ced6
Sep 07 10:21:04.100517 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.100482    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:04.183888 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.183829    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:04.281438 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.281305    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:04.307009 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.306971    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6" event={"ID":"60d763d9-c798-44d1-a3b4-b00feb890428","Type":"ContainerStarted","Data":"a366b7ebf8df893c93a5a35208596ef686d1711fd761811236f7b21dc98256b8"}
Sep 07 10:21:04.311269 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.311237    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x8w2s" event={"ID":"415363e6-edcf-41e5-97ce-cd379cf30f4e","Type":"ContainerStarted","Data":"d75e4228301979112a7446ea1f9a984a22a180fd7a6701f9dc103ff7966e0dc3"}
Sep 07 10:21:04.311414 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.311299    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x8w2s" event={"ID":"415363e6-edcf-41e5-97ce-cd379cf30f4e","Type":"ContainerStarted","Data":"8d39bd587067cd12330b835094e55cf127d7f74deff3bb1fc44613619f9aa2ae"}
Sep 07 10:21:04.313624 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.313590    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5g6z4" event={"ID":"ce8a104b-d12d-46bf-accb-e8ed7d04195c","Type":"ContainerStarted","Data":"96e994af8c3a9ab4461fddbc07604c48823c2589d0bc9c1d2211c9cedbc6ced6"}
Sep 07 10:21:04.315611 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.315569    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7676d8944-tzrbr" event={"ID":"945b76dd-21f5-4690-b0fb-0afbf104601d","Type":"ContainerStarted","Data":"6b0778d73b3474f185c62dd8a8032ea1dfd35267cbccb3e19d97e6d4aae20dde"}
Sep 07 10:21:04.315730 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.315613    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7676d8944-tzrbr" event={"ID":"945b76dd-21f5-4690-b0fb-0afbf104601d","Type":"ContainerStarted","Data":"e654f3a442ef4cf75541f013c62e9782307313c5475d382ad5d737c43de8aa11"}
Sep 07 10:21:04.319218 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.318573    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87d987788-lpz4n" event={"ID":"2284375e-7edc-4b16-b3b6-336d9f25bda4","Type":"ContainerStarted","Data":"c1f9db5855683ee970f281be74d6e7bb4af3f9b12005f43ad4588d975adcdd82"}
Sep 07 10:21:04.319218 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.318647    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87d987788-lpz4n" event={"ID":"2284375e-7edc-4b16-b3b6-336d9f25bda4","Type":"ContainerStarted","Data":"8e5b6cd255cd8d4241c30d5b65425a331298f513d308451d7a1c9eda5a841399"}
Sep 07 10:21:04.319218 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.318864    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:21:04.322062 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.321443    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj" event={"ID":"b38067b2-aa31-4846-b669-8588a67c8fdf","Type":"ContainerStarted","Data":"41b714336c026b06ed3708ca6497c6786eeff1bfb65cba118be738892d636d0f"}
Sep 07 10:21:04.325384 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.324447    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:21:04.327596 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.327575    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:21:04.336455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.336154    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-x8w2s" podStartSLOduration=66.323951596 podStartE2EDuration="1m7.336116493s" totalImagesPullingTime="1.012164897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:04.33361848 +0000 UTC m=+68.067792716" watchObservedRunningTime="2026-09-07 10:21:04.336116493 +0000 UTC m=+68.070290727"
Sep 07 10:21:04.355983 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.355918    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-87d987788-lpz4n" podStartSLOduration=67.355895772 podStartE2EDuration="1m7.355895772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:04.355501247 +0000 UTC m=+68.089675508" watchObservedRunningTime="2026-09-07 10:21:04.355895772 +0000 UTC m=+68.090070000"
Sep 07 10:21:04.363926 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.363839    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:04.383011 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.381396    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7676d8944-tzrbr" podStartSLOduration=56.381366204 podStartE2EDuration="56.381366204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:20:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:04.380988237 +0000 UTC m=+68.115162471" watchObservedRunningTime="2026-09-07 10:21:04.381366204 +0000 UTC m=+68.115540439"
Sep 07 10:21:04.446405 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.446348    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:04.544913 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.544817    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:04.779766 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:04.778596    2577 scope.go:117] "RemoveContainer" containerID="96add42daa5d6cb16c77f6a534f76533778ed728309193ccc1dd0912c558de0c"
Sep 07 10:21:05.066688 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.066349    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:05.333779 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.331951    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:21:05.333779 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.333050    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-grp6m" event={"ID":"72995f6c-69b6-468a-86e2-2471b01e2e4d","Type":"ContainerStarted","Data":"354951ee4c4e5f16120d4f4138a75b417256964956c3f7df43e43582a27ec92f"}
Sep 07 10:21:05.333779 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.333095    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:21:05.333779 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.333681    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:21:05.334251 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.334219    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7676d8944-tzrbr"
Sep 07 10:21:05.350812 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.350752    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-grp6m" podStartSLOduration=45.31088448 podStartE2EDuration="55.350729544s" totalImagesPullingTime="10.039845064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:05.349636902 +0000 UTC m=+69.083811150" watchObservedRunningTime="2026-09-07 10:21:05.350729544 +0000 UTC m=+69.084903779"
Sep 07 10:21:05.459367 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.459307    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:05.513921 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.513886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:05.556811 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.556778    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-grp6m"
Sep 07 10:21:05.754741 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.754712    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-7rng4"]
Sep 07 10:21:05.758196 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.758166    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:05.758444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.758429    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-7rng4"
Sep 07 10:21:05.761091 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.760876    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 07 10:21:05.761410 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.761364    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 07 10:21:05.761565 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.761546    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-l5tjf\""
Sep 07 10:21:05.771791 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.771764    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-7rng4"]
Sep 07 10:21:05.855716 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.855676    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:05.898317 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.898116    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdxdr\" (UniqueName: \"kubernetes.io/projected/c1de07df-6d06-4584-96e2-0bd0f5d5b418-kube-api-access-zdxdr\") pod \"downloads-85d98c998f-7rng4\" (UID: \"c1de07df-6d06-4584-96e2-0bd0f5d5b418\") " pod="openshift-console/downloads-85d98c998f-7rng4"
Sep 07 10:21:05.956406 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.956369    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:05.999493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:05.999384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdxdr\" (UniqueName: \"kubernetes.io/projected/c1de07df-6d06-4584-96e2-0bd0f5d5b418-kube-api-access-zdxdr\") pod \"downloads-85d98c998f-7rng4\" (UID: \"c1de07df-6d06-4584-96e2-0bd0f5d5b418\") " pod="openshift-console/downloads-85d98c998f-7rng4"
Sep 07 10:21:06.009895 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:06.009821    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdxdr\" (UniqueName: \"kubernetes.io/projected/c1de07df-6d06-4584-96e2-0bd0f5d5b418-kube-api-access-zdxdr\") pod \"downloads-85d98c998f-7rng4\" (UID: \"c1de07df-6d06-4584-96e2-0bd0f5d5b418\") " pod="openshift-console/downloads-85d98c998f-7rng4"
Sep 07 10:21:06.056200 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:06.056120    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:06.070999 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:06.070973    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-7rng4"
Sep 07 10:21:06.218297 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:06.218269    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:06.315860 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:06.315735    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:06.605553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:06.605441    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:06.707741 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:06.707706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:06.833095 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:06.833019    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:07.252778 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:07.252731    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:07.436850 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:07.436750    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:07.633834 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:07.633755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:07.686359 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:07.686307    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:07.913553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:07.913465    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:08.013307 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:08.013252    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:08.066177 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:08.066143    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:08.696250 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:08.696223    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-7rng4"]
Sep 07 10:21:08.715670 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:21:08.715640    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1de07df_6d06_4584_96e2_0bd0f5d5b418.slice/crio-005981635b2cfaee81071d422819801799450a4e74ff68df69ad2f2f5b5c5056 WatchSource:0}: Error finding container 005981635b2cfaee81071d422819801799450a4e74ff68df69ad2f2f5b5c5056: Status 404 returned error can't find the container with id 005981635b2cfaee81071d422819801799450a4e74ff68df69ad2f2f5b5c5056
Sep 07 10:21:08.864054 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:08.863958    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:08.964893 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:08.964769    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:09.030790 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.030753    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-scc8r"]
Sep 07 10:21:09.030966 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.030906    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:09.035815 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.035791    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-scc8r"
Sep 07 10:21:09.038238 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.038200    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-488ls\""
Sep 07 10:21:09.038238 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.038227    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 07 10:21:09.043918 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.043894    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-scc8r"]
Sep 07 10:21:09.129745 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.129702    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c2407bec-ba3f-4c44-8908-c0d0b0e72316-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-scc8r\" (UID: \"c2407bec-ba3f-4c44-8908-c0d0b0e72316\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-scc8r"
Sep 07 10:21:09.231345 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.231247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c2407bec-ba3f-4c44-8908-c0d0b0e72316-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-scc8r\" (UID: \"c2407bec-ba3f-4c44-8908-c0d0b0e72316\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-scc8r"
Sep 07 10:21:09.231485 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:21:09.231365    2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 07 10:21:09.231485 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:21:09.231434    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2407bec-ba3f-4c44-8908-c0d0b0e72316-tls-certificates podName:c2407bec-ba3f-4c44-8908-c0d0b0e72316 nodeName:}" failed. No retries permitted until 2026-09-07 10:21:09.731416783 +0000 UTC m=+73.465590994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/c2407bec-ba3f-4c44-8908-c0d0b0e72316-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-scc8r" (UID: "c2407bec-ba3f-4c44-8908-c0d0b0e72316") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 07 10:21:09.347548 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.347512    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-7rng4" event={"ID":"c1de07df-6d06-4584-96e2-0bd0f5d5b418","Type":"ContainerStarted","Data":"005981635b2cfaee81071d422819801799450a4e74ff68df69ad2f2f5b5c5056"}
Sep 07 10:21:09.353732 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.353704    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6" event={"ID":"60d763d9-c798-44d1-a3b4-b00feb890428","Type":"ContainerStarted","Data":"bb97620467f4c81b993dd286b85cd8ef9a825d98394222b46005826ea9c5f77b"}
Sep 07 10:21:09.353904 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.353785    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6" event={"ID":"60d763d9-c798-44d1-a3b4-b00feb890428","Type":"ContainerStarted","Data":"6b45b1d16c61fdbfe084a4b27aab081c6cf4709e596058fddfa38e70c4e3e934"}
Sep 07 10:21:09.355568 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.355545    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5g6z4" event={"ID":"ce8a104b-d12d-46bf-accb-e8ed7d04195c","Type":"ContainerStarted","Data":"f395d5405b822e4366864c7e6e3d83951cda545a32a5eb2642ec3cff733778b4"}
Sep 07 10:21:09.355675 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.355601    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:21:09.355675 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.355624    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5g6z4" event={"ID":"ce8a104b-d12d-46bf-accb-e8ed7d04195c","Type":"ContainerStarted","Data":"e09c2ba8e5495878ad44bb6cd3487a8156683f0a766f5f5ca2376b4415309224"}
Sep 07 10:21:09.357177 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.357154    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ktvqp" event={"ID":"804db42a-4910-4e6f-bdd2-d62cbb720877","Type":"ContainerStarted","Data":"3e8373cb8522608bfeb86f2f870df5d1e562d451c6624a33d722d803ec84885c"}
Sep 07 10:21:09.358625 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.358608    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj" event={"ID":"b38067b2-aa31-4846-b669-8588a67c8fdf","Type":"ContainerStarted","Data":"bf25c9b258145f26436337e12aeeec2502a3739aaa4eec70a442db31b8eede42"}
Sep 07 10:21:09.370842 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.370797    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-vngk6" podStartSLOduration=56.519798396 podStartE2EDuration="1m1.370782228s" totalImagesPullingTime="4.850983832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:09.369448532 +0000 UTC m=+73.103622777" watchObservedRunningTime="2026-09-07 10:21:09.370782228 +0000 UTC m=+73.104956464"
Sep 07 10:21:09.386044 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.386002    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5g6z4" podStartSLOduration=33.621188817 podStartE2EDuration="38.385987931s" totalImagesPullingTime="4.764799114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:20:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:09.384834545 +0000 UTC m=+73.119008777" watchObservedRunningTime="2026-09-07 10:21:09.385987931 +0000 UTC m=+73.120162164"
Sep 07 10:21:09.415124 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.415045    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ktvqp" podStartSLOduration=66.652405771 podStartE2EDuration="1m12.415022848s" totalImagesPullingTime="5.762617077s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 10:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:09.39910957 +0000 UTC m=+73.133283804" watchObservedRunningTime="2026-09-07 10:21:09.415022848 +0000 UTC m=+73.149197081"
Sep 07 10:21:09.415939 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.415909    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-h87jj" podStartSLOduration=56.575390104 podStartE2EDuration="1m1.415898997s" totalImagesPullingTime="4.840508893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:09.4158199 +0000 UTC m=+73.149994133" watchObservedRunningTime="2026-09-07 10:21:09.415898997 +0000 UTC m=+73.150073230"
Sep 07 10:21:09.574773 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.574689    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:09.737083 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.737033    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c2407bec-ba3f-4c44-8908-c0d0b0e72316-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-scc8r\" (UID: \"c2407bec-ba3f-4c44-8908-c0d0b0e72316\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-scc8r"
Sep 07 10:21:09.740002 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.739973    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c2407bec-ba3f-4c44-8908-c0d0b0e72316-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-scc8r\" (UID: \"c2407bec-ba3f-4c44-8908-c0d0b0e72316\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-scc8r"
Sep 07 10:21:09.945586 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:09.945508    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-scc8r"
Sep 07 10:21:10.240184 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:10.240151    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-scc8r"]
Sep 07 10:21:10.243638 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:21:10.243605    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2407bec_ba3f_4c44_8908_c0d0b0e72316.slice/crio-17e4cfe7efc1aad84c90049ee5970cf5352bf6efcb4debf39c202855d7e3e9a7 WatchSource:0}: Error finding container 17e4cfe7efc1aad84c90049ee5970cf5352bf6efcb4debf39c202855d7e3e9a7: Status 404 returned error can't find the container with id 17e4cfe7efc1aad84c90049ee5970cf5352bf6efcb4debf39c202855d7e3e9a7
Sep 07 10:21:10.364675 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:10.364638    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-scc8r" event={"ID":"c2407bec-ba3f-4c44-8908-c0d0b0e72316","Type":"ContainerStarted","Data":"17e4cfe7efc1aad84c90049ee5970cf5352bf6efcb4debf39c202855d7e3e9a7"}
Sep 07 10:21:10.688661 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:10.688569    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:10.741287 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:10.741256    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:11.141218 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:11.141185    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:11.199738 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:11.199696    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:11.807651 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:11.807601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:11.906814 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:11.906784    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:12.376614 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:12.376580    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-scc8r" event={"ID":"c2407bec-ba3f-4c44-8908-c0d0b0e72316","Type":"ContainerStarted","Data":"20a9581281b8059b37a1055873774245acbe51e6bc01f2dc51d122b128557aac"}
Sep 07 10:21:12.376801 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:12.376730    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-scc8r"
Sep 07 10:21:12.382513 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:12.382486    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-scc8r"
Sep 07 10:21:12.391927 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:12.391868    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-scc8r" podStartSLOduration=1.731728385 podStartE2EDuration="3.391847929s" totalImagesPullingTime="1.660119544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:12.389950637 +0000 UTC m=+76.124124873" watchObservedRunningTime="2026-09-07 10:21:12.391847929 +0000 UTC m=+76.126022163"
Sep 07 10:21:12.680271 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:12.680182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:12.732976 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:12.732942    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:13.061750 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:13.061715    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:13.142549 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:13.142511    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:13.468156 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:13.468079    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:15.693343 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:15.693302    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-djsc2\" (UID: \"dc5e172d-81d8-4dd8-9ff7-26010c0ef971\") " pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2"
Sep 07 10:21:15.696288 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:15.696262    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/dc5e172d-81d8-4dd8-9ff7-26010c0ef971-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-djsc2\" (UID: \"dc5e172d-81d8-4dd8-9ff7-26010c0ef971\") " pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2"
Sep 07 10:21:15.834408 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:15.834377    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-8ppc5\""
Sep 07 10:21:15.842493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:15.842463    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2"
Sep 07 10:21:16.132852 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:16.132818    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-djsc2"]
Sep 07 10:21:16.135445 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:21:16.135416    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc5e172d_81d8_4dd8_9ff7_26010c0ef971.slice/crio-d2f61a588f8f7f6c867e3b7351410dc057acecb356a2151f858163e44afa3f4a WatchSource:0}: Error finding container d2f61a588f8f7f6c867e3b7351410dc057acecb356a2151f858163e44afa3f4a: Status 404 returned error can't find the container with id d2f61a588f8f7f6c867e3b7351410dc057acecb356a2151f858163e44afa3f4a
Sep 07 10:21:16.395821 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:16.395728    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2" event={"ID":"dc5e172d-81d8-4dd8-9ff7-26010c0ef971","Type":"ContainerStarted","Data":"d2f61a588f8f7f6c867e3b7351410dc057acecb356a2151f858163e44afa3f4a"}
Sep 07 10:21:17.381023 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:17.380968    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:18.406975 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:18.406937    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2" event={"ID":"dc5e172d-81d8-4dd8-9ff7-26010c0ef971","Type":"ContainerStarted","Data":"35251640da3b35564d41d19a47d6c3559298a6cc8a99c58b508ecd3f909d7354"}
Sep 07 10:21:18.422681 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:18.422615    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-djsc2" podStartSLOduration=34.05206721 podStartE2EDuration="35.422593666s" totalImagesPullingTime="1.370526456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:20:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:18.421386851 +0000 UTC m=+82.155561085" watchObservedRunningTime="2026-09-07 10:21:18.422593666 +0000 UTC m=+82.156767900"
Sep 07 10:21:18.551490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:18.551435    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:18.609049 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:18.609009    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:19.367284 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:19.367253    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5g6z4"
Sep 07 10:21:20.209621 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.209585    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:20.308812 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.308778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:20.409048 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.409016    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:20.485170 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.485062    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hlqhq"]
Sep 07 10:21:20.490032 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.490003    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.493369 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.493343    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 07 10:21:20.493497 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.493346    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8f4ct\""
Sep 07 10:21:20.493497 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.493383    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 07 10:21:20.493973 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.493934    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 07 10:21:20.494071 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.493981    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 07 10:21:20.510049 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.510017    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:20.589223 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.589179    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:20.634116 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.634075    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-node-exporter-tls\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.634322 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.634158    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-node-exporter-wtmp\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.634322 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.634191    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-sys\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.634322 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.634219    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.634322 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.634250    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7s77\" (UniqueName: \"kubernetes.io/projected/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-kube-api-access-k7s77\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.634322 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.634297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-root\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.634563 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.634410    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-metrics-client-ca\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.634563 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.634451    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-node-exporter-textfile\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.634563 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.634487    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.684373 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.684320    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:20.735027 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.734983    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-root\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.735240 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.735040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-metrics-client-ca\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.735240 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.735119    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-root\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.735240 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.735166    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-node-exporter-textfile\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.735240 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.735205    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.735467 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.735248    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-node-exporter-tls\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.735467 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.735275    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-node-exporter-wtmp\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.735467 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.735303    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-sys\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.735467 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.735328    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.735467 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.735356    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7s77\" (UniqueName: \"kubernetes.io/projected/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-kube-api-access-k7s77\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.735710 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.735470    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-node-exporter-textfile\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.735710 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.735579    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-node-exporter-wtmp\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.735710 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.735694    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-metrics-client-ca\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.735858 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.735753    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-sys\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.736015 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.735996    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.738285 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.738265    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-node-exporter-tls\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.738423 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.738402    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.743400 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.743378    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7s77\" (UniqueName: \"kubernetes.io/projected/4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0-kube-api-access-k7s77\") pod \"node-exporter-hlqhq\" (UID: \"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0\") " pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.784332 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.784291    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:20.800479 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.800436    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hlqhq"
Sep 07 10:21:20.985221 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:20.985181    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:21.084868 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:21.084753    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:21.188669 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:21.188634    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:21.286621 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:21.286582    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:21.446806 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:21.446778    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:21.530600 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:21.530506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:21.601607 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:21.601577    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:23.284937 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:23.284901    2577 patch_prober.go:28] interesting pod/image-registry-87d987788-lpz4n container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 07 10:21:23.285387 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:23.284954    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-87d987788-lpz4n" podUID="2284375e-7edc-4b16-b3b6-336d9f25bda4" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 10:21:23.421816 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:23.421776    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:23.522246 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:23.522201    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:23.766520 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:23.766481    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:23.885190 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:23.885143    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:24.894039 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:24.894003    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:24.995040 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:24.995007    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:25.244808 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:25.244711    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:25.263998 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:21:25.263970    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd1ba36_cbaa_4579_84f7_6bd47f81c4c0.slice/crio-c4e5307c3a8206dcdbe2304a273f480019eea2434cdd1c0f86641e39d30a3253 WatchSource:0}: Error finding container c4e5307c3a8206dcdbe2304a273f480019eea2434cdd1c0f86641e39d30a3253: Status 404 returned error can't find the container with id c4e5307c3a8206dcdbe2304a273f480019eea2434cdd1c0f86641e39d30a3253
Sep 07 10:21:25.338094 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:25.338065    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:21:25.338973 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:25.338949    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:25.441446 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:25.441419    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hlqhq" event={"ID":"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0","Type":"ContainerStarted","Data":"c4e5307c3a8206dcdbe2304a273f480019eea2434cdd1c0f86641e39d30a3253"}
Sep 07 10:21:25.487902 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:25.487871    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:25.539164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:25.539059    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:25.595732 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:25.595681    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:26.449081 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:26.449040    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-7rng4" event={"ID":"c1de07df-6d06-4584-96e2-0bd0f5d5b418","Type":"ContainerStarted","Data":"c856564e1c12a81692e95126b9d65d1a3dba90112f6d2cab5789090db77efd16"}
Sep 07 10:21:26.449548 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:26.449378    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-7rng4"
Sep 07 10:21:26.452306 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:26.452276    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hlqhq" event={"ID":"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0","Type":"ContainerStarted","Data":"23006cdd3fe3fb2f74c0e71ab585a5ccb526eea1807f6c3a5d0d31db7de7e30c"}
Sep 07 10:21:26.456604 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:26.456580    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-7rng4"
Sep 07 10:21:26.467967 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:26.467905    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-7rng4" podStartSLOduration=4.835295413 podStartE2EDuration="21.467886212s" totalImagesPullingTime="16.632590799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:26.465651183 +0000 UTC m=+90.199825417" watchObservedRunningTime="2026-09-07 10:21:26.467886212 +0000 UTC m=+90.202060446"
Sep 07 10:21:26.760218 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:26.760166    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:26.805935 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:26.805903    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-87d987788-lpz4n"]
Sep 07 10:21:26.895913 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:26.895852    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:27.459935 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:27.459896    2577 generic.go:356] "Generic (PLEG): container finished" podID="4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0" containerID="23006cdd3fe3fb2f74c0e71ab585a5ccb526eea1807f6c3a5d0d31db7de7e30c" exitCode=0
Sep 07 10:21:27.460450 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:27.459987    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hlqhq" event={"ID":"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0","Type":"ContainerDied","Data":"23006cdd3fe3fb2f74c0e71ab585a5ccb526eea1807f6c3a5d0d31db7de7e30c"}
Sep 07 10:21:27.489819 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:27.489789    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:27.542878 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:27.542843    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:28.468321 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:28.468282    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hlqhq" event={"ID":"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0","Type":"ContainerStarted","Data":"118cc8679bcf318cb1c55ab929079836f80b55a227221cb82a3d6e4505d70100"}
Sep 07 10:21:28.468835 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:28.468343    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hlqhq" event={"ID":"4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0","Type":"ContainerStarted","Data":"532af2819fdebe1e8aab8fb79d808ca950a015e6af3d840da9280c569448bac0"}
Sep 07 10:21:28.488524 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:28.488477    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hlqhq" podStartSLOduration=7.306362593 podStartE2EDuration="8.488460737s" totalImagesPullingTime="1.083104663s" totalInitContainerRuntime="98.993481ms" isStatefulPod=true podCreationTimestamp="2026-09-07 10:21:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:28.486612341 +0000 UTC m=+92.220786574" watchObservedRunningTime="2026-09-07 10:21:28.488460737 +0000 UTC m=+92.222634970"
Sep 07 10:21:29.462851 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.462813    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:29.501297 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.501267    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59fb4f6d85-62dj8"]
Sep 07 10:21:29.538335 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.538301    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59fb4f6d85-62dj8"]
Sep 07 10:21:29.538491 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.538405    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.541524 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.541497    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 07 10:21:29.541676 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.541545    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 07 10:21:29.541676 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.541486    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 07 10:21:29.541861 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.541846    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 07 10:21:29.542526 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.542496    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 07 10:21:29.542749 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.542722    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dflgf\""
Sep 07 10:21:29.546762 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.546510    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 07 10:21:29.559800 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.559772    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:29.615317 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.615281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmsp9\" (UniqueName: \"kubernetes.io/projected/e28ccf68-4116-4721-afb9-5bcedaef9231-kube-api-access-fmsp9\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.615483 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.615330    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e28ccf68-4116-4721-afb9-5bcedaef9231-console-serving-cert\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.615483 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.615377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-trusted-ca-bundle\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.615483 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.615394    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e28ccf68-4116-4721-afb9-5bcedaef9231-console-oauth-config\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.615483 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.615427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-oauth-serving-cert\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.615483 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.615461    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-console-config\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.615707 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.615488    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-service-ca\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.716440 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.716354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e28ccf68-4116-4721-afb9-5bcedaef9231-console-oauth-config\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.716440 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.716416    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-oauth-serving-cert\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.716659 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.716471    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-console-config\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.716659 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.716500    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-service-ca\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.716659 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.716535    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmsp9\" (UniqueName: \"kubernetes.io/projected/e28ccf68-4116-4721-afb9-5bcedaef9231-kube-api-access-fmsp9\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.716659 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.716647    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e28ccf68-4116-4721-afb9-5bcedaef9231-console-serving-cert\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.717297 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.717274    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-oauth-serving-cert\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.717434 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.717327    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-service-ca\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.717434 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.717398    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-trusted-ca-bundle\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.717577 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.717552    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-console-config\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.718220 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.718184    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-trusted-ca-bundle\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.719573 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.719548    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e28ccf68-4116-4721-afb9-5bcedaef9231-console-oauth-config\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.719984 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.719962    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e28ccf68-4116-4721-afb9-5bcedaef9231-console-serving-cert\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.724890 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.724866    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmsp9\" (UniqueName: \"kubernetes.io/projected/e28ccf68-4116-4721-afb9-5bcedaef9231-kube-api-access-fmsp9\") pod \"console-59fb4f6d85-62dj8\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") " pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:29.850041 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:29.850007    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:30.071108 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:30.071013    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:30.165416 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:30.165383    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59fb4f6d85-62dj8"]
Sep 07 10:21:30.168076 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:30.168044    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:30.169773 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:21:30.169743    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28ccf68_4116_4721_afb9_5bcedaef9231.slice/crio-74372ddd5b7fb51abb7320580f0ae565787f489243d9311792fb04641871ea48 WatchSource:0}: Error finding container 74372ddd5b7fb51abb7320580f0ae565787f489243d9311792fb04641871ea48: Status 404 returned error can't find the container with id 74372ddd5b7fb51abb7320580f0ae565787f489243d9311792fb04641871ea48
Sep 07 10:21:30.478533 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:30.478489    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59fb4f6d85-62dj8" event={"ID":"e28ccf68-4116-4721-afb9-5bcedaef9231","Type":"ContainerStarted","Data":"74372ddd5b7fb51abb7320580f0ae565787f489243d9311792fb04641871ea48"}
Sep 07 10:21:30.663208 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:30.663173    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:30.718844 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:30.718802    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:31.672159 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:31.672055    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:31.733467 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:31.733423    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:32.001574 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:32.001503    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:32.106155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:32.106101    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:32.203902 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:32.203853    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:32.304277 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:32.304162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:33.879970 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:33.879941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:33.932804 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:33.932775    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:34.324618 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:34.324585    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9xlk4"
Sep 07 10:21:34.499927 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:34.499893    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59fb4f6d85-62dj8" event={"ID":"e28ccf68-4116-4721-afb9-5bcedaef9231","Type":"ContainerStarted","Data":"9945ea62ffe4115b1d27d8f1bd8178be1756bbfca4f7ce5024fb13817fd9f148"}
Sep 07 10:21:34.523373 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:34.523313    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59fb4f6d85-62dj8" podStartSLOduration=1.865995546 podStartE2EDuration="5.523291497s" totalImagesPullingTime="3.657295951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:21:34.52137764 +0000 UTC m=+98.255551873" watchObservedRunningTime="2026-09-07 10:21:34.523291497 +0000 UTC m=+98.257465733"
Sep 07 10:21:35.125582 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:35.125549    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:35.226045 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:35.226006    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:37.916147 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:37.916101    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:38.015987 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:38.015955    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:39.850333 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:39.850283    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:39.850789 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:39.850355    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:39.855351 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:39.855330    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:40.372023 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:40.371989    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:40.428399 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:40.428348    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:40.525680 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:40.525652    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:21:40.686844 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:40.686765    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:40.789140 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:40.789089    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:40.891723 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:40.891688    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:41.010188 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:41.010147    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:41.110665 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:41.110629    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:42.901538 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:42.901498    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:42.963114 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:42.963063    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:50.698555 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:50.698518    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:51.831765 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:51.831729    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-87d987788-lpz4n" podUID="2284375e-7edc-4b16-b3b6-336d9f25bda4" containerName="registry" containerID="cri-o://c1f9db5855683ee970f281be74d6e7bb4af3f9b12005f43ad4588d975adcdd82" gracePeriod=30
Sep 07 10:21:52.068106 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.068085    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:21:52.104176 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.104106    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls\") pod \"2284375e-7edc-4b16-b3b6-336d9f25bda4\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") "
Sep 07 10:21:52.104176 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.104154    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2284375e-7edc-4b16-b3b6-336d9f25bda4-installation-pull-secrets\") pod \"2284375e-7edc-4b16-b3b6-336d9f25bda4\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") "
Sep 07 10:21:52.104359 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.104179    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-bound-sa-token\") pod \"2284375e-7edc-4b16-b3b6-336d9f25bda4\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") "
Sep 07 10:21:52.104359 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.104228    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2284375e-7edc-4b16-b3b6-336d9f25bda4-ca-trust-extracted\") pod \"2284375e-7edc-4b16-b3b6-336d9f25bda4\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") "
Sep 07 10:21:52.104359 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.104262    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2284375e-7edc-4b16-b3b6-336d9f25bda4-image-registry-private-configuration\") pod \"2284375e-7edc-4b16-b3b6-336d9f25bda4\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") "
Sep 07 10:21:52.104359 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.104291    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4rf5\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-kube-api-access-h4rf5\") pod \"2284375e-7edc-4b16-b3b6-336d9f25bda4\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") "
Sep 07 10:21:52.104359 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.104352    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2284375e-7edc-4b16-b3b6-336d9f25bda4-trusted-ca\") pod \"2284375e-7edc-4b16-b3b6-336d9f25bda4\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") "
Sep 07 10:21:52.105347 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.105307    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2284375e-7edc-4b16-b3b6-336d9f25bda4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2284375e-7edc-4b16-b3b6-336d9f25bda4" (UID: "2284375e-7edc-4b16-b3b6-336d9f25bda4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:21:52.105470 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.105420    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-certificates\") pod \"2284375e-7edc-4b16-b3b6-336d9f25bda4\" (UID: \"2284375e-7edc-4b16-b3b6-336d9f25bda4\") "
Sep 07 10:21:52.105884 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.105859    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2284375e-7edc-4b16-b3b6-336d9f25bda4" (UID: "2284375e-7edc-4b16-b3b6-336d9f25bda4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:21:52.106164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.106118    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2284375e-7edc-4b16-b3b6-336d9f25bda4-trusted-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:21:52.106262 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.106172    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-certificates\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:21:52.107016 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.106935    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2284375e-7edc-4b16-b3b6-336d9f25bda4" (UID: "2284375e-7edc-4b16-b3b6-336d9f25bda4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:21:52.107232 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.107187    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2284375e-7edc-4b16-b3b6-336d9f25bda4-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2284375e-7edc-4b16-b3b6-336d9f25bda4" (UID: "2284375e-7edc-4b16-b3b6-336d9f25bda4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:21:52.107343 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.107268    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2284375e-7edc-4b16-b3b6-336d9f25bda4" (UID: "2284375e-7edc-4b16-b3b6-336d9f25bda4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:21:52.107402 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.107362    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2284375e-7edc-4b16-b3b6-336d9f25bda4-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2284375e-7edc-4b16-b3b6-336d9f25bda4" (UID: "2284375e-7edc-4b16-b3b6-336d9f25bda4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:21:52.108036 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.108009    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-kube-api-access-h4rf5" (OuterVolumeSpecName: "kube-api-access-h4rf5") pod "2284375e-7edc-4b16-b3b6-336d9f25bda4" (UID: "2284375e-7edc-4b16-b3b6-336d9f25bda4"). InnerVolumeSpecName "kube-api-access-h4rf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:21:52.115586 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.115562    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2284375e-7edc-4b16-b3b6-336d9f25bda4-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2284375e-7edc-4b16-b3b6-336d9f25bda4" (UID: "2284375e-7edc-4b16-b3b6-336d9f25bda4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:21:52.207447 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.207413    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-registry-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:21:52.207447 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.207439    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2284375e-7edc-4b16-b3b6-336d9f25bda4-installation-pull-secrets\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:21:52.207447 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.207448    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-bound-sa-token\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:21:52.207447 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.207457    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2284375e-7edc-4b16-b3b6-336d9f25bda4-ca-trust-extracted\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:21:52.207702 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.207466    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2284375e-7edc-4b16-b3b6-336d9f25bda4-image-registry-private-configuration\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:21:52.207702 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.207475    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4rf5\" (UniqueName: \"kubernetes.io/projected/2284375e-7edc-4b16-b3b6-336d9f25bda4-kube-api-access-h4rf5\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:21:52.569892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.569858    2577 generic.go:356] "Generic (PLEG): container finished" podID="2284375e-7edc-4b16-b3b6-336d9f25bda4" containerID="c1f9db5855683ee970f281be74d6e7bb4af3f9b12005f43ad4588d975adcdd82" exitCode=0
Sep 07 10:21:52.570053 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.569905    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87d987788-lpz4n"
Sep 07 10:21:52.570053 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.569939    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87d987788-lpz4n" event={"ID":"2284375e-7edc-4b16-b3b6-336d9f25bda4","Type":"ContainerDied","Data":"c1f9db5855683ee970f281be74d6e7bb4af3f9b12005f43ad4588d975adcdd82"}
Sep 07 10:21:52.570053 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.569986    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87d987788-lpz4n" event={"ID":"2284375e-7edc-4b16-b3b6-336d9f25bda4","Type":"ContainerDied","Data":"8e5b6cd255cd8d4241c30d5b65425a331298f513d308451d7a1c9eda5a841399"}
Sep 07 10:21:52.570053 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.570005    2577 scope.go:117] "RemoveContainer" containerID="c1f9db5855683ee970f281be74d6e7bb4af3f9b12005f43ad4588d975adcdd82"
Sep 07 10:21:52.578683 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.578668    2577 scope.go:117] "RemoveContainer" containerID="c1f9db5855683ee970f281be74d6e7bb4af3f9b12005f43ad4588d975adcdd82"
Sep 07 10:21:52.578950 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:21:52.578932    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1f9db5855683ee970f281be74d6e7bb4af3f9b12005f43ad4588d975adcdd82\": container with ID starting with c1f9db5855683ee970f281be74d6e7bb4af3f9b12005f43ad4588d975adcdd82 not found: ID does not exist" containerID="c1f9db5855683ee970f281be74d6e7bb4af3f9b12005f43ad4588d975adcdd82"
Sep 07 10:21:52.578996 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.578957    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1f9db5855683ee970f281be74d6e7bb4af3f9b12005f43ad4588d975adcdd82"} err="failed to get container status \"c1f9db5855683ee970f281be74d6e7bb4af3f9b12005f43ad4588d975adcdd82\": rpc error: code = NotFound desc = could not find container \"c1f9db5855683ee970f281be74d6e7bb4af3f9b12005f43ad4588d975adcdd82\": container with ID starting with c1f9db5855683ee970f281be74d6e7bb4af3f9b12005f43ad4588d975adcdd82 not found: ID does not exist"
Sep 07 10:21:52.590249 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.590212    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-87d987788-lpz4n"]
Sep 07 10:21:52.593780 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.593756    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-87d987788-lpz4n"]
Sep 07 10:21:52.783041 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.783004    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2284375e-7edc-4b16-b3b6-336d9f25bda4" path="/var/lib/kubelet/pods/2284375e-7edc-4b16-b3b6-336d9f25bda4/volumes"
Sep 07 10:21:52.915212 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.915175    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:52.963726 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:52.963698    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:21:53.575787 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:53.575756    2577 generic.go:356] "Generic (PLEG): container finished" podID="1ad9111f-27a6-4f1f-b886-66c18049ccb6" containerID="ef0ed5fcd1af90a7eb2f00e5df6d3be9408eedc3d15d1581864d2fd4b974c5d5" exitCode=0
Sep 07 10:21:53.575941 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:53.575829    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn" event={"ID":"1ad9111f-27a6-4f1f-b886-66c18049ccb6","Type":"ContainerDied","Data":"ef0ed5fcd1af90a7eb2f00e5df6d3be9408eedc3d15d1581864d2fd4b974c5d5"}
Sep 07 10:21:53.576175 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:53.576152    2577 scope.go:117] "RemoveContainer" containerID="ef0ed5fcd1af90a7eb2f00e5df6d3be9408eedc3d15d1581864d2fd4b974c5d5"
Sep 07 10:21:54.582976 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:21:54.582943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-8rktn" event={"ID":"1ad9111f-27a6-4f1f-b886-66c18049ccb6","Type":"ContainerStarted","Data":"04eca0ec0cbf64e17d81f884b8ca68da4337622f734e404bba90fc1088026ac4"}
Sep 07 10:22:03.464047 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:22:03.464009    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:22:05.874002 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:22:05.873957    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:22:05.977611 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:22:05.977576    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:22:13.652544 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:22:13.652463    2577 generic.go:356] "Generic (PLEG): container finished" podID="228ba212-8e2a-4dab-8381-cc40eb711df7" containerID="a0ce011bb9c59ff614df2375abae6b88b8367888d87c99eadaa586e3197e71f7" exitCode=0
Sep 07 10:22:13.653015 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:22:13.652538    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt" event={"ID":"228ba212-8e2a-4dab-8381-cc40eb711df7","Type":"ContainerDied","Data":"a0ce011bb9c59ff614df2375abae6b88b8367888d87c99eadaa586e3197e71f7"}
Sep 07 10:22:13.653015 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:22:13.652922    2577 scope.go:117] "RemoveContainer" containerID="a0ce011bb9c59ff614df2375abae6b88b8367888d87c99eadaa586e3197e71f7"
Sep 07 10:22:13.654124 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:22:13.654105    2577 generic.go:356] "Generic (PLEG): container finished" podID="bee903a3-a8e5-4120-9a8f-573d186cdf2c" containerID="0be39ffce509388c1a4eee12b8aa78188bb0fe5e07475bfba4038f5b705af940" exitCode=0
Sep 07 10:22:13.654222 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:22:13.654155    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn" event={"ID":"bee903a3-a8e5-4120-9a8f-573d186cdf2c","Type":"ContainerDied","Data":"0be39ffce509388c1a4eee12b8aa78188bb0fe5e07475bfba4038f5b705af940"}
Sep 07 10:22:13.654417 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:22:13.654404    2577 scope.go:117] "RemoveContainer" containerID="0be39ffce509388c1a4eee12b8aa78188bb0fe5e07475bfba4038f5b705af940"
Sep 07 10:22:14.661817 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:22:14.661782    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-srlkt" event={"ID":"228ba212-8e2a-4dab-8381-cc40eb711df7","Type":"ContainerStarted","Data":"f6af5167a883341914101437fa533294743293cf1ba84eb22b43a607b92735ef"}
Sep 07 10:22:14.663893 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:22:14.663866    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-zbhwn" event={"ID":"bee903a3-a8e5-4120-9a8f-573d186cdf2c","Type":"ContainerStarted","Data":"625cc27a2fe2a5eab9a5d21adac874b6612dd572980682426c736b61a10229d2"}
Sep 07 10:22:22.645228 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:22:22.645171    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:22:22.704253 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:22:22.704214    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:22:37.928955 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:22:37.928914    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:22:38.029796 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:22:38.029760    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:23:01.704323 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.704228    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bd5f49fbb-kl4mh"]
Sep 07 10:23:01.704868 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.704666    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2284375e-7edc-4b16-b3b6-336d9f25bda4" containerName="registry"
Sep 07 10:23:01.704868 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.704685    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2284375e-7edc-4b16-b3b6-336d9f25bda4" containerName="registry"
Sep 07 10:23:01.704868 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.704778    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2284375e-7edc-4b16-b3b6-336d9f25bda4" containerName="registry"
Sep 07 10:23:01.707801 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.707780    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.741008 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.740970    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bd5f49fbb-kl4mh"]
Sep 07 10:23:01.753666 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.753632    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-config\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.753822 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.753694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-service-ca\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.753822 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.753720    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-serving-cert\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.753822 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.753744    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-trusted-ca-bundle\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.753822 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.753774    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-oauth-config\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.753822 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.753795    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-oauth-serving-cert\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.754026 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.753881    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsw6m\" (UniqueName: \"kubernetes.io/projected/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-kube-api-access-fsw6m\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.795108 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.795066    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:23:01.855357 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.855318    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-config\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.855541 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.855382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-service-ca\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.855541 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.855408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-serving-cert\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.855541 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.855436    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-trusted-ca-bundle\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.855541 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.855468    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-oauth-config\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.855541 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.855493    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-oauth-serving-cert\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.855541 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.855536    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsw6m\" (UniqueName: \"kubernetes.io/projected/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-kube-api-access-fsw6m\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.856176 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.856148    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-config\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.856470 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.856301    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-service-ca\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.856470 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.856301    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-oauth-serving-cert\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.856595 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.856565    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-trusted-ca-bundle\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.858239 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.858213    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-oauth-config\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.858331 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.858272    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-serving-cert\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.863223 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.863196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsw6m\" (UniqueName: \"kubernetes.io/projected/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-kube-api-access-fsw6m\") pod \"console-bd5f49fbb-kl4mh\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") " pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:01.895635 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:01.895600    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:23:02.017871 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:02.017786    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:02.290364 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:02.290281    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bd5f49fbb-kl4mh"]
Sep 07 10:23:02.292547 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:23:02.292504    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5704ff9e_8cde_4c3b_ae73_785a05f1f2d8.slice/crio-0ddc50e6d4ea7e87d02a7a546fca5d6af686886248199b4d09e262104f809c06 WatchSource:0}: Error finding container 0ddc50e6d4ea7e87d02a7a546fca5d6af686886248199b4d09e262104f809c06: Status 404 returned error can't find the container with id 0ddc50e6d4ea7e87d02a7a546fca5d6af686886248199b4d09e262104f809c06
Sep 07 10:23:02.846211 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:02.846173    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bd5f49fbb-kl4mh" event={"ID":"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8","Type":"ContainerStarted","Data":"0864c98f6db809b456ece16dfebbaf4939bf174c81e4a1736594f6571bc1b5b8"}
Sep 07 10:23:02.846576 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:02.846231    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bd5f49fbb-kl4mh" event={"ID":"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8","Type":"ContainerStarted","Data":"0ddc50e6d4ea7e87d02a7a546fca5d6af686886248199b4d09e262104f809c06"}
Sep 07 10:23:02.866143 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:02.866077    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bd5f49fbb-kl4mh" podStartSLOduration=1.866059113 podStartE2EDuration="1.866059113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:23:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:23:02.86489643 +0000 UTC m=+186.599070673" watchObservedRunningTime="2026-09-07 10:23:02.866059113 +0000 UTC m=+186.600233346"
Sep 07 10:23:12.017974 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:12.017929    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:12.018464 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:12.018011    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:12.022715 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:12.022692    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:12.886285 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:12.886254    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:23:12.934228 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:12.934184    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59fb4f6d85-62dj8"]
Sep 07 10:23:13.044394 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:13.044367    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:23:13.094165 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:13.094118    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:23:13.145296 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:13.145199    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:23:14.046847 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:14.046809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:23:37.957672 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:37.957606    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59fb4f6d85-62dj8" podUID="e28ccf68-4116-4721-afb9-5bcedaef9231" containerName="console" containerID="cri-o://9945ea62ffe4115b1d27d8f1bd8178be1756bbfca4f7ce5024fb13817fd9f148" gracePeriod=15
Sep 07 10:23:38.200983 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.200958    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:23:38.263004 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.262916    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e28ccf68-4116-4721-afb9-5bcedaef9231-console-serving-cert\") pod \"e28ccf68-4116-4721-afb9-5bcedaef9231\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") "
Sep 07 10:23:38.263004 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.262961    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-trusted-ca-bundle\") pod \"e28ccf68-4116-4721-afb9-5bcedaef9231\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") "
Sep 07 10:23:38.263004 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.262990    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e28ccf68-4116-4721-afb9-5bcedaef9231-console-oauth-config\") pod \"e28ccf68-4116-4721-afb9-5bcedaef9231\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") "
Sep 07 10:23:38.263304 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.263028    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmsp9\" (UniqueName: \"kubernetes.io/projected/e28ccf68-4116-4721-afb9-5bcedaef9231-kube-api-access-fmsp9\") pod \"e28ccf68-4116-4721-afb9-5bcedaef9231\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") "
Sep 07 10:23:38.263304 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.263081    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-service-ca\") pod \"e28ccf68-4116-4721-afb9-5bcedaef9231\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") "
Sep 07 10:23:38.263304 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.263104    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-oauth-serving-cert\") pod \"e28ccf68-4116-4721-afb9-5bcedaef9231\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") "
Sep 07 10:23:38.263304 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.263158    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-console-config\") pod \"e28ccf68-4116-4721-afb9-5bcedaef9231\" (UID: \"e28ccf68-4116-4721-afb9-5bcedaef9231\") "
Sep 07 10:23:38.263501 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.263474    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e28ccf68-4116-4721-afb9-5bcedaef9231" (UID: "e28ccf68-4116-4721-afb9-5bcedaef9231"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:23:38.263501 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.263480    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-service-ca" (OuterVolumeSpecName: "service-ca") pod "e28ccf68-4116-4721-afb9-5bcedaef9231" (UID: "e28ccf68-4116-4721-afb9-5bcedaef9231"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:23:38.263646 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.263615    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e28ccf68-4116-4721-afb9-5bcedaef9231" (UID: "e28ccf68-4116-4721-afb9-5bcedaef9231"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:23:38.263703 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.263677    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-console-config" (OuterVolumeSpecName: "console-config") pod "e28ccf68-4116-4721-afb9-5bcedaef9231" (UID: "e28ccf68-4116-4721-afb9-5bcedaef9231"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:23:38.265528 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.265494    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e28ccf68-4116-4721-afb9-5bcedaef9231-kube-api-access-fmsp9" (OuterVolumeSpecName: "kube-api-access-fmsp9") pod "e28ccf68-4116-4721-afb9-5bcedaef9231" (UID: "e28ccf68-4116-4721-afb9-5bcedaef9231"). InnerVolumeSpecName "kube-api-access-fmsp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:23:38.265769 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.265743    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e28ccf68-4116-4721-afb9-5bcedaef9231-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e28ccf68-4116-4721-afb9-5bcedaef9231" (UID: "e28ccf68-4116-4721-afb9-5bcedaef9231"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:23:38.265769 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.265753    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e28ccf68-4116-4721-afb9-5bcedaef9231-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e28ccf68-4116-4721-afb9-5bcedaef9231" (UID: "e28ccf68-4116-4721-afb9-5bcedaef9231"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:23:38.364479 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.364424    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e28ccf68-4116-4721-afb9-5bcedaef9231-console-serving-cert\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:23:38.364479 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.364472    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-trusted-ca-bundle\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:23:38.364479 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.364485    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e28ccf68-4116-4721-afb9-5bcedaef9231-console-oauth-config\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:23:38.364736 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.364500    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmsp9\" (UniqueName: \"kubernetes.io/projected/e28ccf68-4116-4721-afb9-5bcedaef9231-kube-api-access-fmsp9\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:23:38.364736 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.364513    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:23:38.364736 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.364523    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-oauth-serving-cert\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:23:38.364736 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.364534    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e28ccf68-4116-4721-afb9-5bcedaef9231-console-config\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:23:38.981762 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.981730    2577 generic.go:356] "Generic (PLEG): container finished" podID="e28ccf68-4116-4721-afb9-5bcedaef9231" containerID="9945ea62ffe4115b1d27d8f1bd8178be1756bbfca4f7ce5024fb13817fd9f148" exitCode=0
Sep 07 10:23:38.982159 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.981771    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59fb4f6d85-62dj8"
Sep 07 10:23:38.982159 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.981783    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59fb4f6d85-62dj8" event={"ID":"e28ccf68-4116-4721-afb9-5bcedaef9231","Type":"ContainerDied","Data":"9945ea62ffe4115b1d27d8f1bd8178be1756bbfca4f7ce5024fb13817fd9f148"}
Sep 07 10:23:38.982159 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.981824    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59fb4f6d85-62dj8" event={"ID":"e28ccf68-4116-4721-afb9-5bcedaef9231","Type":"ContainerDied","Data":"74372ddd5b7fb51abb7320580f0ae565787f489243d9311792fb04641871ea48"}
Sep 07 10:23:38.982159 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.981842    2577 scope.go:117] "RemoveContainer" containerID="9945ea62ffe4115b1d27d8f1bd8178be1756bbfca4f7ce5024fb13817fd9f148"
Sep 07 10:23:38.990282 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.990264    2577 scope.go:117] "RemoveContainer" containerID="9945ea62ffe4115b1d27d8f1bd8178be1756bbfca4f7ce5024fb13817fd9f148"
Sep 07 10:23:38.990546 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:23:38.990525    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9945ea62ffe4115b1d27d8f1bd8178be1756bbfca4f7ce5024fb13817fd9f148\": container with ID starting with 9945ea62ffe4115b1d27d8f1bd8178be1756bbfca4f7ce5024fb13817fd9f148 not found: ID does not exist" containerID="9945ea62ffe4115b1d27d8f1bd8178be1756bbfca4f7ce5024fb13817fd9f148"
Sep 07 10:23:38.990600 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.990553    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9945ea62ffe4115b1d27d8f1bd8178be1756bbfca4f7ce5024fb13817fd9f148"} err="failed to get container status \"9945ea62ffe4115b1d27d8f1bd8178be1756bbfca4f7ce5024fb13817fd9f148\": rpc error: code = NotFound desc = could not find container \"9945ea62ffe4115b1d27d8f1bd8178be1756bbfca4f7ce5024fb13817fd9f148\": container with ID starting with 9945ea62ffe4115b1d27d8f1bd8178be1756bbfca4f7ce5024fb13817fd9f148 not found: ID does not exist"
Sep 07 10:23:38.998042 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.998020    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59fb4f6d85-62dj8"]
Sep 07 10:23:38.999749 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:38.999729    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59fb4f6d85-62dj8"]
Sep 07 10:23:40.783653 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:23:40.783623    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e28ccf68-4116-4721-afb9-5bcedaef9231" path="/var/lib/kubelet/pods/e28ccf68-4116-4721-afb9-5bcedaef9231/volumes"
Sep 07 10:24:52.633206 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:24:52.633166    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:24:55.181407 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:24:55.181368    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:24:55.231696 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:24:55.231655    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:24:56.684217 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:24:56.684188    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:24:56.694621 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:24:56.694598    2577 kubelet.go:1648] "Image garbage collection succeeded"
Sep 07 10:24:56.694872 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:24:56.694858    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:25:05.821392 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:25:05.821353    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:25:05.895448 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:25:05.895418    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:25:07.771671 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:25:07.771633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:26:20.646494 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.646406    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56c94f66c8-4qf99"]
Sep 07 10:26:20.646995 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.646805    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e28ccf68-4116-4721-afb9-5bcedaef9231" containerName="console"
Sep 07 10:26:20.646995 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.646820    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e28ccf68-4116-4721-afb9-5bcedaef9231" containerName="console"
Sep 07 10:26:20.646995 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.646907    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e28ccf68-4116-4721-afb9-5bcedaef9231" containerName="console"
Sep 07 10:26:20.649950 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.649927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.667001 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.666976    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56c94f66c8-4qf99"]
Sep 07 10:26:20.699804 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.699764    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:26:20.754065 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.754025    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:26:20.788468 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.788437    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-oauth-serving-cert\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.788468 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.788467    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmnvq\" (UniqueName: \"kubernetes.io/projected/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-kube-api-access-rmnvq\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.788777 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.788494    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-console-config\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.788777 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.788647    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-trusted-ca-bundle\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.788777 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.788692    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-console-oauth-config\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.788777 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.788723    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-console-serving-cert\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.788777 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.788754    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-service-ca\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.889320 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.889283    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-trusted-ca-bundle\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.889320 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.889322    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-console-oauth-config\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.889524 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.889342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-console-serving-cert\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.889524 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.889361    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-service-ca\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.889524 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.889379    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-oauth-serving-cert\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.889524 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.889392    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmnvq\" (UniqueName: \"kubernetes.io/projected/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-kube-api-access-rmnvq\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.889524 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.889420    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-console-config\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.890220 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.890188    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-service-ca\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.890327 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.890190    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-oauth-serving-cert\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.890327 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.890308    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-trusted-ca-bundle\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.890461 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.890440    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-console-config\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.891943 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.891925    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-console-oauth-config\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.892027 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.892011    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-console-serving-cert\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.897862 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.897803    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmnvq\" (UniqueName: \"kubernetes.io/projected/a5299f7c-f528-4b86-9f30-5e13df5f3c9a-kube-api-access-rmnvq\") pod \"console-56c94f66c8-4qf99\" (UID: \"a5299f7c-f528-4b86-9f30-5e13df5f3c9a\") " pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:20.960567 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:20.960531    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:21.232067 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:21.232034    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56c94f66c8-4qf99"]
Sep 07 10:26:21.235206 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:26:21.235178    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5299f7c_f528_4b86_9f30_5e13df5f3c9a.slice/crio-fdf14e6a18bee30e72ce49dac7b7efe4da8ffb554c1785f4a0e56b4b755ff505 WatchSource:0}: Error finding container fdf14e6a18bee30e72ce49dac7b7efe4da8ffb554c1785f4a0e56b4b755ff505: Status 404 returned error can't find the container with id fdf14e6a18bee30e72ce49dac7b7efe4da8ffb554c1785f4a0e56b4b755ff505
Sep 07 10:26:21.600220 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:21.600093    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56c94f66c8-4qf99" event={"ID":"a5299f7c-f528-4b86-9f30-5e13df5f3c9a","Type":"ContainerStarted","Data":"7f6cad3def701a8cdd66842bdf9b5acbfdf45fe93ab9602c600f469f00afe35a"}
Sep 07 10:26:21.600220 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:21.600185    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56c94f66c8-4qf99" event={"ID":"a5299f7c-f528-4b86-9f30-5e13df5f3c9a","Type":"ContainerStarted","Data":"fdf14e6a18bee30e72ce49dac7b7efe4da8ffb554c1785f4a0e56b4b755ff505"}
Sep 07 10:26:21.618610 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:21.618555    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56c94f66c8-4qf99" podStartSLOduration=1.6185396939999999 podStartE2EDuration="1.618539694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:26:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:26:21.617703405 +0000 UTC m=+385.351877637" watchObservedRunningTime="2026-09-07 10:26:21.618539694 +0000 UTC m=+385.352714017"
Sep 07 10:26:30.961654 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:30.961611    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:30.962180 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:30.961691    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:30.966807 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:30.966782    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:31.643796 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:31.643768    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56c94f66c8-4qf99"
Sep 07 10:26:31.690184 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:31.690121    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bd5f49fbb-kl4mh"]
Sep 07 10:26:31.798090 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:31.798057    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:26:31.899565 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:31.899469    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:26:31.953151 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:31.953078    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:26:56.718694 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:56.718629    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bd5f49fbb-kl4mh" podUID="5704ff9e-8cde-4c3b-ae73-785a05f1f2d8" containerName="console" containerID="cri-o://0864c98f6db809b456ece16dfebbaf4939bf174c81e4a1736594f6571bc1b5b8" gracePeriod=15
Sep 07 10:26:56.969249 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:56.969175    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:26:57.063045 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.063012    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-oauth-serving-cert\") pod \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") "
Sep 07 10:26:57.063045 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.063065    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-service-ca\") pod \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") "
Sep 07 10:26:57.063356 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.063107    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-trusted-ca-bundle\") pod \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") "
Sep 07 10:26:57.063356 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.063153    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-config\") pod \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") "
Sep 07 10:26:57.063356 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.063179    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-serving-cert\") pod \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") "
Sep 07 10:26:57.063356 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.063215    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsw6m\" (UniqueName: \"kubernetes.io/projected/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-kube-api-access-fsw6m\") pod \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") "
Sep 07 10:26:57.063356 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.063239    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-oauth-config\") pod \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\" (UID: \"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8\") "
Sep 07 10:26:57.063678 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.063650    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5704ff9e-8cde-4c3b-ae73-785a05f1f2d8" (UID: "5704ff9e-8cde-4c3b-ae73-785a05f1f2d8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:26:57.063762 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.063676    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5704ff9e-8cde-4c3b-ae73-785a05f1f2d8" (UID: "5704ff9e-8cde-4c3b-ae73-785a05f1f2d8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:26:57.063762 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.063699    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-service-ca" (OuterVolumeSpecName: "service-ca") pod "5704ff9e-8cde-4c3b-ae73-785a05f1f2d8" (UID: "5704ff9e-8cde-4c3b-ae73-785a05f1f2d8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:26:57.063857 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.063779    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-config" (OuterVolumeSpecName: "console-config") pod "5704ff9e-8cde-4c3b-ae73-785a05f1f2d8" (UID: "5704ff9e-8cde-4c3b-ae73-785a05f1f2d8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:26:57.065752 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.065719    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5704ff9e-8cde-4c3b-ae73-785a05f1f2d8" (UID: "5704ff9e-8cde-4c3b-ae73-785a05f1f2d8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:26:57.065852 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.065768    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-kube-api-access-fsw6m" (OuterVolumeSpecName: "kube-api-access-fsw6m") pod "5704ff9e-8cde-4c3b-ae73-785a05f1f2d8" (UID: "5704ff9e-8cde-4c3b-ae73-785a05f1f2d8"). InnerVolumeSpecName "kube-api-access-fsw6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:26:57.065852 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.065785    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5704ff9e-8cde-4c3b-ae73-785a05f1f2d8" (UID: "5704ff9e-8cde-4c3b-ae73-785a05f1f2d8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:26:57.164547 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.164498    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-trusted-ca-bundle\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:26:57.164547 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.164541    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-config\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:26:57.164547 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.164551    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-serving-cert\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:26:57.164547 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.164560    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsw6m\" (UniqueName: \"kubernetes.io/projected/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-kube-api-access-fsw6m\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:26:57.164827 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.164568    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-console-oauth-config\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:26:57.164827 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.164579    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-oauth-serving-cert\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:26:57.164827 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.164587    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:26:57.737856 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.737819    2577 generic.go:356] "Generic (PLEG): container finished" podID="5704ff9e-8cde-4c3b-ae73-785a05f1f2d8" containerID="0864c98f6db809b456ece16dfebbaf4939bf174c81e4a1736594f6571bc1b5b8" exitCode=0
Sep 07 10:26:57.738302 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.737866    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bd5f49fbb-kl4mh"
Sep 07 10:26:57.738302 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.737897    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bd5f49fbb-kl4mh" event={"ID":"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8","Type":"ContainerDied","Data":"0864c98f6db809b456ece16dfebbaf4939bf174c81e4a1736594f6571bc1b5b8"}
Sep 07 10:26:57.738302 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.737947    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bd5f49fbb-kl4mh" event={"ID":"5704ff9e-8cde-4c3b-ae73-785a05f1f2d8","Type":"ContainerDied","Data":"0ddc50e6d4ea7e87d02a7a546fca5d6af686886248199b4d09e262104f809c06"}
Sep 07 10:26:57.738302 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.737967    2577 scope.go:117] "RemoveContainer" containerID="0864c98f6db809b456ece16dfebbaf4939bf174c81e4a1736594f6571bc1b5b8"
Sep 07 10:26:57.746919 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.746899    2577 scope.go:117] "RemoveContainer" containerID="0864c98f6db809b456ece16dfebbaf4939bf174c81e4a1736594f6571bc1b5b8"
Sep 07 10:26:57.747277 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:26:57.747259    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0864c98f6db809b456ece16dfebbaf4939bf174c81e4a1736594f6571bc1b5b8\": container with ID starting with 0864c98f6db809b456ece16dfebbaf4939bf174c81e4a1736594f6571bc1b5b8 not found: ID does not exist" containerID="0864c98f6db809b456ece16dfebbaf4939bf174c81e4a1736594f6571bc1b5b8"
Sep 07 10:26:57.747343 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.747286    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0864c98f6db809b456ece16dfebbaf4939bf174c81e4a1736594f6571bc1b5b8"} err="failed to get container status \"0864c98f6db809b456ece16dfebbaf4939bf174c81e4a1736594f6571bc1b5b8\": rpc error: code = NotFound desc = could not find container \"0864c98f6db809b456ece16dfebbaf4939bf174c81e4a1736594f6571bc1b5b8\": container with ID starting with 0864c98f6db809b456ece16dfebbaf4939bf174c81e4a1736594f6571bc1b5b8 not found: ID does not exist"
Sep 07 10:26:57.767000 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.766956    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bd5f49fbb-kl4mh"]
Sep 07 10:26:57.771664 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:57.771632    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bd5f49fbb-kl4mh"]
Sep 07 10:26:58.784259 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:26:58.784219    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5704ff9e-8cde-4c3b-ae73-785a05f1f2d8" path="/var/lib/kubelet/pods/5704ff9e-8cde-4c3b-ae73-785a05f1f2d8/volumes"
Sep 07 10:27:47.887409 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:27:47.887350    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:28:06.583498 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:28:06.583457    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:28:06.708388 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:28:06.708347    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:29:10.210341 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:29:10.210302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:29:56.709548 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:29:56.709518    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:29:56.723591 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:29:56.723555    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:30:28.288348 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:28.288269    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:30:28.973933 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:28.973901    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"]
Sep 07 10:30:28.974215 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:28.974203    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5704ff9e-8cde-4c3b-ae73-785a05f1f2d8" containerName="console"
Sep 07 10:30:28.974270 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:28.974217    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5704ff9e-8cde-4c3b-ae73-785a05f1f2d8" containerName="console"
Sep 07 10:30:28.974304 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:28.974276    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5704ff9e-8cde-4c3b-ae73-785a05f1f2d8" containerName="console"
Sep 07 10:30:28.977328 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:28.977312    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"
Sep 07 10:30:28.980023 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:28.979999    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 07 10:30:28.981196 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:28.981176    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9cdh5\""
Sep 07 10:30:28.981302 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:28.981213    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 07 10:30:28.994547 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:28.994520    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"]
Sep 07 10:30:29.090937 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:29.090896    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86404fa3-515a-4fde-aad1-449a4a888200-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c\" (UID: \"86404fa3-515a-4fde-aad1-449a4a888200\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"
Sep 07 10:30:29.091107 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:29.090959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86404fa3-515a-4fde-aad1-449a4a888200-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c\" (UID: \"86404fa3-515a-4fde-aad1-449a4a888200\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"
Sep 07 10:30:29.091107 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:29.090995    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvv4j\" (UniqueName: \"kubernetes.io/projected/86404fa3-515a-4fde-aad1-449a4a888200-kube-api-access-qvv4j\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c\" (UID: \"86404fa3-515a-4fde-aad1-449a4a888200\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"
Sep 07 10:30:29.191985 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:29.191953    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvv4j\" (UniqueName: \"kubernetes.io/projected/86404fa3-515a-4fde-aad1-449a4a888200-kube-api-access-qvv4j\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c\" (UID: \"86404fa3-515a-4fde-aad1-449a4a888200\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"
Sep 07 10:30:29.192125 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:29.192007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86404fa3-515a-4fde-aad1-449a4a888200-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c\" (UID: \"86404fa3-515a-4fde-aad1-449a4a888200\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"
Sep 07 10:30:29.192125 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:29.192036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86404fa3-515a-4fde-aad1-449a4a888200-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c\" (UID: \"86404fa3-515a-4fde-aad1-449a4a888200\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"
Sep 07 10:30:29.192373 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:29.192358    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86404fa3-515a-4fde-aad1-449a4a888200-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c\" (UID: \"86404fa3-515a-4fde-aad1-449a4a888200\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"
Sep 07 10:30:29.192453 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:29.192436    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86404fa3-515a-4fde-aad1-449a4a888200-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c\" (UID: \"86404fa3-515a-4fde-aad1-449a4a888200\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"
Sep 07 10:30:29.201253 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:29.201224    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvv4j\" (UniqueName: \"kubernetes.io/projected/86404fa3-515a-4fde-aad1-449a4a888200-kube-api-access-qvv4j\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c\" (UID: \"86404fa3-515a-4fde-aad1-449a4a888200\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"
Sep 07 10:30:29.286265 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:29.286167    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"
Sep 07 10:30:29.439462 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:29.439428    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:30:29.560238 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:29.560190    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"]
Sep 07 10:30:29.562913 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:30:29.562889    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86404fa3_515a_4fde_aad1_449a4a888200.slice/crio-47683b2149d346d2313815115c9f63a187668d58eab673eba20804800fb28b28 WatchSource:0}: Error finding container 47683b2149d346d2313815115c9f63a187668d58eab673eba20804800fb28b28: Status 404 returned error can't find the container with id 47683b2149d346d2313815115c9f63a187668d58eab673eba20804800fb28b28
Sep 07 10:30:29.564902 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:29.564886    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 10:30:30.544975 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:30.544939    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c" event={"ID":"86404fa3-515a-4fde-aad1-449a4a888200","Type":"ContainerStarted","Data":"47683b2149d346d2313815115c9f63a187668d58eab673eba20804800fb28b28"}
Sep 07 10:30:36.571807 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:36.571764    2577 generic.go:356] "Generic (PLEG): container finished" podID="86404fa3-515a-4fde-aad1-449a4a888200" containerID="11c769db723f8a8ff20ad41e331669a42966cef2753729cead02bf7c5ab067f8" exitCode=0
Sep 07 10:30:36.571807 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:36.571812    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c" event={"ID":"86404fa3-515a-4fde-aad1-449a4a888200","Type":"ContainerDied","Data":"11c769db723f8a8ff20ad41e331669a42966cef2753729cead02bf7c5ab067f8"}
Sep 07 10:30:39.586865 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:39.586834    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c" event={"ID":"86404fa3-515a-4fde-aad1-449a4a888200","Type":"ContainerStarted","Data":"f85fcceeafcd769bfde7c5af7b7b9eb76c91a3b1b0671f4ee3cdc59d8de2f7ae"}
Sep 07 10:30:40.592970 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:40.592934    2577 generic.go:356] "Generic (PLEG): container finished" podID="86404fa3-515a-4fde-aad1-449a4a888200" containerID="f85fcceeafcd769bfde7c5af7b7b9eb76c91a3b1b0671f4ee3cdc59d8de2f7ae" exitCode=0
Sep 07 10:30:40.592970 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:40.592973    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c" event={"ID":"86404fa3-515a-4fde-aad1-449a4a888200","Type":"ContainerDied","Data":"f85fcceeafcd769bfde7c5af7b7b9eb76c91a3b1b0671f4ee3cdc59d8de2f7ae"}
Sep 07 10:30:48.629341 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:48.629310    2577 generic.go:356] "Generic (PLEG): container finished" podID="86404fa3-515a-4fde-aad1-449a4a888200" containerID="3b363648e71159d186fd4455fd47de1dd84b382a017153aa91b6151f3950fe8a" exitCode=0
Sep 07 10:30:48.629700 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:48.629395    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c" event={"ID":"86404fa3-515a-4fde-aad1-449a4a888200","Type":"ContainerDied","Data":"3b363648e71159d186fd4455fd47de1dd84b382a017153aa91b6151f3950fe8a"}
Sep 07 10:30:49.762592 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:49.762568    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"
Sep 07 10:30:49.860249 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:49.860220    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86404fa3-515a-4fde-aad1-449a4a888200-bundle\") pod \"86404fa3-515a-4fde-aad1-449a4a888200\" (UID: \"86404fa3-515a-4fde-aad1-449a4a888200\") "
Sep 07 10:30:49.860412 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:49.860278    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86404fa3-515a-4fde-aad1-449a4a888200-util\") pod \"86404fa3-515a-4fde-aad1-449a4a888200\" (UID: \"86404fa3-515a-4fde-aad1-449a4a888200\") "
Sep 07 10:30:49.860412 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:49.860305    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvv4j\" (UniqueName: \"kubernetes.io/projected/86404fa3-515a-4fde-aad1-449a4a888200-kube-api-access-qvv4j\") pod \"86404fa3-515a-4fde-aad1-449a4a888200\" (UID: \"86404fa3-515a-4fde-aad1-449a4a888200\") "
Sep 07 10:30:49.860842 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:49.860821    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86404fa3-515a-4fde-aad1-449a4a888200-bundle" (OuterVolumeSpecName: "bundle") pod "86404fa3-515a-4fde-aad1-449a4a888200" (UID: "86404fa3-515a-4fde-aad1-449a4a888200"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:30:49.862657 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:49.862631    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86404fa3-515a-4fde-aad1-449a4a888200-kube-api-access-qvv4j" (OuterVolumeSpecName: "kube-api-access-qvv4j") pod "86404fa3-515a-4fde-aad1-449a4a888200" (UID: "86404fa3-515a-4fde-aad1-449a4a888200"). InnerVolumeSpecName "kube-api-access-qvv4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:30:49.864750 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:49.864731    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86404fa3-515a-4fde-aad1-449a4a888200-util" (OuterVolumeSpecName: "util") pod "86404fa3-515a-4fde-aad1-449a4a888200" (UID: "86404fa3-515a-4fde-aad1-449a4a888200"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:30:49.961444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:49.961417    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86404fa3-515a-4fde-aad1-449a4a888200-bundle\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:30:49.961444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:49.961442    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86404fa3-515a-4fde-aad1-449a4a888200-util\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:30:49.961610 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:49.961455    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvv4j\" (UniqueName: \"kubernetes.io/projected/86404fa3-515a-4fde-aad1-449a4a888200-kube-api-access-qvv4j\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:30:50.639190 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:50.639160    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c"
Sep 07 10:30:50.639365 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:50.639155    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vh48c" event={"ID":"86404fa3-515a-4fde-aad1-449a4a888200","Type":"ContainerDied","Data":"47683b2149d346d2313815115c9f63a187668d58eab673eba20804800fb28b28"}
Sep 07 10:30:50.639365 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:50.639285    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47683b2149d346d2313815115c9f63a187668d58eab673eba20804800fb28b28"
Sep 07 10:30:55.031660 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:55.031615    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:30:55.871109 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:55.871074    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:30:56.925924 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:56.925890    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzdzz"]
Sep 07 10:30:56.926412 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:56.926235    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86404fa3-515a-4fde-aad1-449a4a888200" containerName="util"
Sep 07 10:30:56.926412 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:56.926251    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="86404fa3-515a-4fde-aad1-449a4a888200" containerName="util"
Sep 07 10:30:56.926412 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:56.926263    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86404fa3-515a-4fde-aad1-449a4a888200" containerName="extract"
Sep 07 10:30:56.926412 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:56.926269    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="86404fa3-515a-4fde-aad1-449a4a888200" containerName="extract"
Sep 07 10:30:56.926412 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:56.926291    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86404fa3-515a-4fde-aad1-449a4a888200" containerName="pull"
Sep 07 10:30:56.926412 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:56.926297    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="86404fa3-515a-4fde-aad1-449a4a888200" containerName="pull"
Sep 07 10:30:56.926412 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:56.926354    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="86404fa3-515a-4fde-aad1-449a4a888200" containerName="extract"
Sep 07 10:30:56.932876 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:56.932855    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzdzz"
Sep 07 10:30:56.935554 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:56.935535    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 07 10:30:56.935673 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:56.935536    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 07 10:30:56.935800 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:56.935788    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-b6mg2\""
Sep 07 10:30:56.947808 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:56.947785    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzdzz"]
Sep 07 10:30:57.010588 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:57.010553    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7a7c8de-8b6a-4d85-9e4b-c706ce3e3470-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vzdzz\" (UID: \"c7a7c8de-8b6a-4d85-9e4b-c706ce3e3470\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzdzz"
Sep 07 10:30:57.010754 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:57.010635    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpnhv\" (UniqueName: \"kubernetes.io/projected/c7a7c8de-8b6a-4d85-9e4b-c706ce3e3470-kube-api-access-rpnhv\") pod \"cert-manager-operator-controller-manager-55c444bf86-vzdzz\" (UID: \"c7a7c8de-8b6a-4d85-9e4b-c706ce3e3470\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzdzz"
Sep 07 10:30:57.110973 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:57.110942    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpnhv\" (UniqueName: \"kubernetes.io/projected/c7a7c8de-8b6a-4d85-9e4b-c706ce3e3470-kube-api-access-rpnhv\") pod \"cert-manager-operator-controller-manager-55c444bf86-vzdzz\" (UID: \"c7a7c8de-8b6a-4d85-9e4b-c706ce3e3470\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzdzz"
Sep 07 10:30:57.111177 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:57.110992    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7a7c8de-8b6a-4d85-9e4b-c706ce3e3470-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vzdzz\" (UID: \"c7a7c8de-8b6a-4d85-9e4b-c706ce3e3470\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzdzz"
Sep 07 10:30:57.111331 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:57.111314    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7a7c8de-8b6a-4d85-9e4b-c706ce3e3470-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vzdzz\" (UID: \"c7a7c8de-8b6a-4d85-9e4b-c706ce3e3470\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzdzz"
Sep 07 10:30:57.120838 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:57.120819    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpnhv\" (UniqueName: \"kubernetes.io/projected/c7a7c8de-8b6a-4d85-9e4b-c706ce3e3470-kube-api-access-rpnhv\") pod \"cert-manager-operator-controller-manager-55c444bf86-vzdzz\" (UID: \"c7a7c8de-8b6a-4d85-9e4b-c706ce3e3470\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzdzz"
Sep 07 10:30:57.241978 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:57.241942    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzdzz"
Sep 07 10:30:57.518039 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:57.518016    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzdzz"]
Sep 07 10:30:57.520300 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:30:57.520274    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7a7c8de_8b6a_4d85_9e4b_c706ce3e3470.slice/crio-c8582d31fe768aac4aa49d357bbfbad9ce67147c349cad7533f6d84e421cc21a WatchSource:0}: Error finding container c8582d31fe768aac4aa49d357bbfbad9ce67147c349cad7533f6d84e421cc21a: Status 404 returned error can't find the container with id c8582d31fe768aac4aa49d357bbfbad9ce67147c349cad7533f6d84e421cc21a
Sep 07 10:30:57.669534 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:57.669496    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzdzz" event={"ID":"c7a7c8de-8b6a-4d85-9e4b-c706ce3e3470","Type":"ContainerStarted","Data":"c8582d31fe768aac4aa49d357bbfbad9ce67147c349cad7533f6d84e421cc21a"}
Sep 07 10:30:58.637707 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:58.637658    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:30:58.739649 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:58.739618    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:30:59.686776 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:59.686696    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzdzz" event={"ID":"c7a7c8de-8b6a-4d85-9e4b-c706ce3e3470","Type":"ContainerStarted","Data":"9637871ca0f5efac9f2a7956a17b45b977629095ad3a10ca9398b9577c290bd6"}
Sep 07 10:30:59.709662 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:30:59.709617    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vzdzz" podStartSLOduration=2.022677139 podStartE2EDuration="3.70960147s" totalImagesPullingTime="1.686924331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:30:59.707347234 +0000 UTC m=+663.441521466" watchObservedRunningTime="2026-09-07 10:30:59.70960147 +0000 UTC m=+663.443775702"
Sep 07 10:31:00.096597 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.096559    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:00.226367 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.226330    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:00.656703 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.656665    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"]
Sep 07 10:31:00.660268 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.660245    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"
Sep 07 10:31:00.662992 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.662971    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 07 10:31:00.663868 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.663842    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 07 10:31:00.663868 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.663864    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9cdh5\""
Sep 07 10:31:00.670573 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.670553    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"]
Sep 07 10:31:00.737531 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.737506    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3cc4f76-d353-4c32-aecb-b63619739ad6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g\" (UID: \"b3cc4f76-d353-4c32-aecb-b63619739ad6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"
Sep 07 10:31:00.737827 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.737549    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwbxg\" (UniqueName: \"kubernetes.io/projected/b3cc4f76-d353-4c32-aecb-b63619739ad6-kube-api-access-nwbxg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g\" (UID: \"b3cc4f76-d353-4c32-aecb-b63619739ad6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"
Sep 07 10:31:00.737827 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.737581    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3cc4f76-d353-4c32-aecb-b63619739ad6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g\" (UID: \"b3cc4f76-d353-4c32-aecb-b63619739ad6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"
Sep 07 10:31:00.838635 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.838608    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwbxg\" (UniqueName: \"kubernetes.io/projected/b3cc4f76-d353-4c32-aecb-b63619739ad6-kube-api-access-nwbxg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g\" (UID: \"b3cc4f76-d353-4c32-aecb-b63619739ad6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"
Sep 07 10:31:00.838759 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.838647    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3cc4f76-d353-4c32-aecb-b63619739ad6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g\" (UID: \"b3cc4f76-d353-4c32-aecb-b63619739ad6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"
Sep 07 10:31:00.838759 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.838728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3cc4f76-d353-4c32-aecb-b63619739ad6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g\" (UID: \"b3cc4f76-d353-4c32-aecb-b63619739ad6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"
Sep 07 10:31:00.839071 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.839052    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3cc4f76-d353-4c32-aecb-b63619739ad6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g\" (UID: \"b3cc4f76-d353-4c32-aecb-b63619739ad6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"
Sep 07 10:31:00.839171 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.839086    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3cc4f76-d353-4c32-aecb-b63619739ad6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g\" (UID: \"b3cc4f76-d353-4c32-aecb-b63619739ad6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"
Sep 07 10:31:00.848523 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.848476    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwbxg\" (UniqueName: \"kubernetes.io/projected/b3cc4f76-d353-4c32-aecb-b63619739ad6-kube-api-access-nwbxg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g\" (UID: \"b3cc4f76-d353-4c32-aecb-b63619739ad6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"
Sep 07 10:31:00.920487 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.920424    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:00.969321 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:00.969296    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"
Sep 07 10:31:01.022640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:01.022582    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:01.253153 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:01.253102    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"]
Sep 07 10:31:01.255903 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:31:01.255874    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3cc4f76_d353_4c32_aecb_b63619739ad6.slice/crio-7fc27dfc3e3fd4c5e7012c38f28df45ffe7d7e6dee485d0ff2c4708b70fb0ff3 WatchSource:0}: Error finding container 7fc27dfc3e3fd4c5e7012c38f28df45ffe7d7e6dee485d0ff2c4708b70fb0ff3: Status 404 returned error can't find the container with id 7fc27dfc3e3fd4c5e7012c38f28df45ffe7d7e6dee485d0ff2c4708b70fb0ff3
Sep 07 10:31:01.697812 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:01.697783    2577 generic.go:356] "Generic (PLEG): container finished" podID="b3cc4f76-d353-4c32-aecb-b63619739ad6" containerID="ad1199c8fb37a342fab29eac77c360f1e6660df0a39fcdb0746d76ac17a71b73" exitCode=0
Sep 07 10:31:01.697969 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:01.697854    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g" event={"ID":"b3cc4f76-d353-4c32-aecb-b63619739ad6","Type":"ContainerDied","Data":"ad1199c8fb37a342fab29eac77c360f1e6660df0a39fcdb0746d76ac17a71b73"}
Sep 07 10:31:01.697969 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:01.697923    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g" event={"ID":"b3cc4f76-d353-4c32-aecb-b63619739ad6","Type":"ContainerStarted","Data":"7fc27dfc3e3fd4c5e7012c38f28df45ffe7d7e6dee485d0ff2c4708b70fb0ff3"}
Sep 07 10:31:03.138799 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:03.138765    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:03.332421 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:03.332388    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:03.511648 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:03.511619    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lxkrg"]
Sep 07 10:31:03.514888 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:03.514867    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lxkrg"
Sep 07 10:31:03.517358 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:03.517327    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-54zsm\""
Sep 07 10:31:03.517460 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:03.517378    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 07 10:31:03.518514 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:03.518496    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 07 10:31:03.521801 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:03.521780    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lxkrg"]
Sep 07 10:31:03.558323 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:03.558301    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e61c8e85-d9e5-4146-ac4b-4cdb69259cd6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lxkrg\" (UID: \"e61c8e85-d9e5-4146-ac4b-4cdb69259cd6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lxkrg"
Sep 07 10:31:03.558446 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:03.558337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v7wr\" (UniqueName: \"kubernetes.io/projected/e61c8e85-d9e5-4146-ac4b-4cdb69259cd6-kube-api-access-6v7wr\") pod \"cert-manager-cainjector-559cdd8c87-lxkrg\" (UID: \"e61c8e85-d9e5-4146-ac4b-4cdb69259cd6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lxkrg"
Sep 07 10:31:03.659142 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:03.659109    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e61c8e85-d9e5-4146-ac4b-4cdb69259cd6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lxkrg\" (UID: \"e61c8e85-d9e5-4146-ac4b-4cdb69259cd6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lxkrg"
Sep 07 10:31:03.659287 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:03.659180    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v7wr\" (UniqueName: \"kubernetes.io/projected/e61c8e85-d9e5-4146-ac4b-4cdb69259cd6-kube-api-access-6v7wr\") pod \"cert-manager-cainjector-559cdd8c87-lxkrg\" (UID: \"e61c8e85-d9e5-4146-ac4b-4cdb69259cd6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lxkrg"
Sep 07 10:31:03.667893 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:03.667868    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e61c8e85-d9e5-4146-ac4b-4cdb69259cd6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lxkrg\" (UID: \"e61c8e85-d9e5-4146-ac4b-4cdb69259cd6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lxkrg"
Sep 07 10:31:03.668216 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:03.668195    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v7wr\" (UniqueName: \"kubernetes.io/projected/e61c8e85-d9e5-4146-ac4b-4cdb69259cd6-kube-api-access-6v7wr\") pod \"cert-manager-cainjector-559cdd8c87-lxkrg\" (UID: \"e61c8e85-d9e5-4146-ac4b-4cdb69259cd6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lxkrg"
Sep 07 10:31:03.834983 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:03.834901    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lxkrg"
Sep 07 10:31:04.712307 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:04.712278    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lxkrg"]
Sep 07 10:31:04.713302 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:04.713278    2577 generic.go:356] "Generic (PLEG): container finished" podID="b3cc4f76-d353-4c32-aecb-b63619739ad6" containerID="733ad5b489c725e7f7b1c48a0e0b1720354ad10fc6db985416053d0b6db14a6c" exitCode=0
Sep 07 10:31:04.713403 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:04.713334    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g" event={"ID":"b3cc4f76-d353-4c32-aecb-b63619739ad6","Type":"ContainerDied","Data":"733ad5b489c725e7f7b1c48a0e0b1720354ad10fc6db985416053d0b6db14a6c"}
Sep 07 10:31:04.714810 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:31:04.714735    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode61c8e85_d9e5_4146_ac4b_4cdb69259cd6.slice/crio-e843dce5ffca68cc38b3f7b7918a3e1316b29ac096b002793fcc054d706aefe2 WatchSource:0}: Error finding container e843dce5ffca68cc38b3f7b7918a3e1316b29ac096b002793fcc054d706aefe2: Status 404 returned error can't find the container with id e843dce5ffca68cc38b3f7b7918a3e1316b29ac096b002793fcc054d706aefe2
Sep 07 10:31:05.422829 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:05.422800    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:05.480831 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:05.480780    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:05.719696 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:05.719654    2577 generic.go:356] "Generic (PLEG): container finished" podID="b3cc4f76-d353-4c32-aecb-b63619739ad6" containerID="8c57c7af2f8026968ae79e6a7e17ff4f14edc7675d3c55f0c899e2b77c712f09" exitCode=0
Sep 07 10:31:05.720121 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:05.719730    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g" event={"ID":"b3cc4f76-d353-4c32-aecb-b63619739ad6","Type":"ContainerDied","Data":"8c57c7af2f8026968ae79e6a7e17ff4f14edc7675d3c55f0c899e2b77c712f09"}
Sep 07 10:31:05.720978 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:05.720955    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lxkrg" event={"ID":"e61c8e85-d9e5-4146-ac4b-4cdb69259cd6","Type":"ContainerStarted","Data":"e843dce5ffca68cc38b3f7b7918a3e1316b29ac096b002793fcc054d706aefe2"}
Sep 07 10:31:06.875379 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:06.875352    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"
Sep 07 10:31:06.886502 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:06.886477    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwbxg\" (UniqueName: \"kubernetes.io/projected/b3cc4f76-d353-4c32-aecb-b63619739ad6-kube-api-access-nwbxg\") pod \"b3cc4f76-d353-4c32-aecb-b63619739ad6\" (UID: \"b3cc4f76-d353-4c32-aecb-b63619739ad6\") "
Sep 07 10:31:06.886608 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:06.886570    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3cc4f76-d353-4c32-aecb-b63619739ad6-util\") pod \"b3cc4f76-d353-4c32-aecb-b63619739ad6\" (UID: \"b3cc4f76-d353-4c32-aecb-b63619739ad6\") "
Sep 07 10:31:06.886708 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:06.886689    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3cc4f76-d353-4c32-aecb-b63619739ad6-bundle\") pod \"b3cc4f76-d353-4c32-aecb-b63619739ad6\" (UID: \"b3cc4f76-d353-4c32-aecb-b63619739ad6\") "
Sep 07 10:31:06.887317 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:06.887268    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3cc4f76-d353-4c32-aecb-b63619739ad6-bundle" (OuterVolumeSpecName: "bundle") pod "b3cc4f76-d353-4c32-aecb-b63619739ad6" (UID: "b3cc4f76-d353-4c32-aecb-b63619739ad6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:31:06.889060 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:06.889025    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3cc4f76-d353-4c32-aecb-b63619739ad6-kube-api-access-nwbxg" (OuterVolumeSpecName: "kube-api-access-nwbxg") pod "b3cc4f76-d353-4c32-aecb-b63619739ad6" (UID: "b3cc4f76-d353-4c32-aecb-b63619739ad6"). InnerVolumeSpecName "kube-api-access-nwbxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:31:06.893940 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:06.893913    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3cc4f76-d353-4c32-aecb-b63619739ad6-util" (OuterVolumeSpecName: "util") pod "b3cc4f76-d353-4c32-aecb-b63619739ad6" (UID: "b3cc4f76-d353-4c32-aecb-b63619739ad6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:31:06.987569 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:06.987494    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b3cc4f76-d353-4c32-aecb-b63619739ad6-util\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:31:06.987569 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:06.987525    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b3cc4f76-d353-4c32-aecb-b63619739ad6-bundle\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:31:06.987569 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:06.987534    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwbxg\" (UniqueName: \"kubernetes.io/projected/b3cc4f76-d353-4c32-aecb-b63619739ad6-kube-api-access-nwbxg\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:31:07.733109 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:07.732897    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g"
Sep 07 10:31:07.733109 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:07.732927    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fbqj8g" event={"ID":"b3cc4f76-d353-4c32-aecb-b63619739ad6","Type":"ContainerDied","Data":"7fc27dfc3e3fd4c5e7012c38f28df45ffe7d7e6dee485d0ff2c4708b70fb0ff3"}
Sep 07 10:31:07.733109 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:07.732973    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fc27dfc3e3fd4c5e7012c38f28df45ffe7d7e6dee485d0ff2c4708b70fb0ff3"
Sep 07 10:31:08.740684 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:08.740650    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lxkrg" event={"ID":"e61c8e85-d9e5-4146-ac4b-4cdb69259cd6","Type":"ContainerStarted","Data":"044b675d2af4411ae8b8f08c78cd66f21012bf5faafd94a966099c7c4ccb7d6d"}
Sep 07 10:31:08.760067 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:08.760016    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lxkrg" podStartSLOduration=2.466026854 podStartE2EDuration="5.760001782s" totalImagesPullingTime="3.293974928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:31:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:31:08.759003859 +0000 UTC m=+672.493178092" watchObservedRunningTime="2026-09-07 10:31:08.760001782 +0000 UTC m=+672.494176014"
Sep 07 10:31:08.790010 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:08.789981    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:08.891147 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:08.891107    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:11.335610 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:11.335577    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:11.440976 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:11.440937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:11.540300 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:11.540264    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:13.579855 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.579821    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xh7w5"]
Sep 07 10:31:13.580217 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.580090    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3cc4f76-d353-4c32-aecb-b63619739ad6" containerName="util"
Sep 07 10:31:13.580217 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.580101    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3cc4f76-d353-4c32-aecb-b63619739ad6" containerName="util"
Sep 07 10:31:13.580217 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.580114    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3cc4f76-d353-4c32-aecb-b63619739ad6" containerName="extract"
Sep 07 10:31:13.580217 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.580121    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3cc4f76-d353-4c32-aecb-b63619739ad6" containerName="extract"
Sep 07 10:31:13.580217 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.580142    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3cc4f76-d353-4c32-aecb-b63619739ad6" containerName="pull"
Sep 07 10:31:13.580217 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.580149    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3cc4f76-d353-4c32-aecb-b63619739ad6" containerName="pull"
Sep 07 10:31:13.580217 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.580210    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3cc4f76-d353-4c32-aecb-b63619739ad6" containerName="extract"
Sep 07 10:31:13.583217 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.583200    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xh7w5"
Sep 07 10:31:13.585575 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.585553    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-fwsf6\""
Sep 07 10:31:13.594413 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.594392    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xh7w5"]
Sep 07 10:31:13.635874 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.635855    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwrl5\" (UniqueName: \"kubernetes.io/projected/eba0b2a5-27da-4a8b-9336-cf5d9694b5b9-kube-api-access-pwrl5\") pod \"cert-manager-55b4d954d5-xh7w5\" (UID: \"eba0b2a5-27da-4a8b-9336-cf5d9694b5b9\") " pod="cert-manager/cert-manager-55b4d954d5-xh7w5"
Sep 07 10:31:13.635962 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.635900    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eba0b2a5-27da-4a8b-9336-cf5d9694b5b9-bound-sa-token\") pod \"cert-manager-55b4d954d5-xh7w5\" (UID: \"eba0b2a5-27da-4a8b-9336-cf5d9694b5b9\") " pod="cert-manager/cert-manager-55b4d954d5-xh7w5"
Sep 07 10:31:13.736424 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.736400    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwrl5\" (UniqueName: \"kubernetes.io/projected/eba0b2a5-27da-4a8b-9336-cf5d9694b5b9-kube-api-access-pwrl5\") pod \"cert-manager-55b4d954d5-xh7w5\" (UID: \"eba0b2a5-27da-4a8b-9336-cf5d9694b5b9\") " pod="cert-manager/cert-manager-55b4d954d5-xh7w5"
Sep 07 10:31:13.736529 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.736442    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eba0b2a5-27da-4a8b-9336-cf5d9694b5b9-bound-sa-token\") pod \"cert-manager-55b4d954d5-xh7w5\" (UID: \"eba0b2a5-27da-4a8b-9336-cf5d9694b5b9\") " pod="cert-manager/cert-manager-55b4d954d5-xh7w5"
Sep 07 10:31:13.746569 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.746544    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eba0b2a5-27da-4a8b-9336-cf5d9694b5b9-bound-sa-token\") pod \"cert-manager-55b4d954d5-xh7w5\" (UID: \"eba0b2a5-27da-4a8b-9336-cf5d9694b5b9\") " pod="cert-manager/cert-manager-55b4d954d5-xh7w5"
Sep 07 10:31:13.746686 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.746588    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwrl5\" (UniqueName: \"kubernetes.io/projected/eba0b2a5-27da-4a8b-9336-cf5d9694b5b9-kube-api-access-pwrl5\") pod \"cert-manager-55b4d954d5-xh7w5\" (UID: \"eba0b2a5-27da-4a8b-9336-cf5d9694b5b9\") " pod="cert-manager/cert-manager-55b4d954d5-xh7w5"
Sep 07 10:31:13.891812 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:13.891751    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xh7w5"
Sep 07 10:31:14.159967 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:14.159943    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xh7w5"]
Sep 07 10:31:14.162560 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:31:14.162532    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeba0b2a5_27da_4a8b_9336_cf5d9694b5b9.slice/crio-a2a87687f97ac59431e70f69101f3c636fcda59855ddd5c0bcc4afce8bc6157c WatchSource:0}: Error finding container a2a87687f97ac59431e70f69101f3c636fcda59855ddd5c0bcc4afce8bc6157c: Status 404 returned error can't find the container with id a2a87687f97ac59431e70f69101f3c636fcda59855ddd5c0bcc4afce8bc6157c
Sep 07 10:31:14.768525 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:14.768488    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xh7w5" event={"ID":"eba0b2a5-27da-4a8b-9336-cf5d9694b5b9","Type":"ContainerStarted","Data":"3e9ba2e1a134e9626efef491ee53c1e93082db9ffc22150a2bfd014883edac13"}
Sep 07 10:31:14.768525 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:14.768531    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xh7w5" event={"ID":"eba0b2a5-27da-4a8b-9336-cf5d9694b5b9","Type":"ContainerStarted","Data":"a2a87687f97ac59431e70f69101f3c636fcda59855ddd5c0bcc4afce8bc6157c"}
Sep 07 10:31:14.787066 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:14.787015    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-xh7w5" podStartSLOduration=1.786999276 podStartE2EDuration="1.786999276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:31:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:31:14.785286029 +0000 UTC m=+678.519465197" watchObservedRunningTime="2026-09-07 10:31:14.786999276 +0000 UTC m=+678.521173508"
Sep 07 10:31:21.693198 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:21.693161    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:21.788406 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:21.788366    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:22.009370 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:22.009331    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:23.838711 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:23.838674    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"]
Sep 07 10:31:23.841670 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:23.841649    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"
Sep 07 10:31:23.844305 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:23.844283    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 07 10:31:23.845387 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:23.845370    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 07 10:31:23.845488 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:23.845466    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9cdh5\""
Sep 07 10:31:23.850544 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:23.850520    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"]
Sep 07 10:31:23.919300 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:23.919261    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfjv5\" (UniqueName: \"kubernetes.io/projected/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-kube-api-access-xfjv5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t\" (UID: \"8bb4d11f-408e-4979-b4b4-d4b80316d7c2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"
Sep 07 10:31:23.919300 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:23.919302    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t\" (UID: \"8bb4d11f-408e-4979-b4b4-d4b80316d7c2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"
Sep 07 10:31:23.919519 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:23.919332    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t\" (UID: \"8bb4d11f-408e-4979-b4b4-d4b80316d7c2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"
Sep 07 10:31:24.020081 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:24.020048    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t\" (UID: \"8bb4d11f-408e-4979-b4b4-d4b80316d7c2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"
Sep 07 10:31:24.020317 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:24.020180    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfjv5\" (UniqueName: \"kubernetes.io/projected/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-kube-api-access-xfjv5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t\" (UID: \"8bb4d11f-408e-4979-b4b4-d4b80316d7c2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"
Sep 07 10:31:24.020317 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:24.020216    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t\" (UID: \"8bb4d11f-408e-4979-b4b4-d4b80316d7c2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"
Sep 07 10:31:24.020490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:24.020468    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t\" (UID: \"8bb4d11f-408e-4979-b4b4-d4b80316d7c2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"
Sep 07 10:31:24.020571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:24.020551    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t\" (UID: \"8bb4d11f-408e-4979-b4b4-d4b80316d7c2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"
Sep 07 10:31:24.028943 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:24.028913    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfjv5\" (UniqueName: \"kubernetes.io/projected/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-kube-api-access-xfjv5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t\" (UID: \"8bb4d11f-408e-4979-b4b4-d4b80316d7c2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"
Sep 07 10:31:24.152728 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:24.152638    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"
Sep 07 10:31:24.422151 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:24.422050    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"]
Sep 07 10:31:24.425056 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:31:24.425025    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bb4d11f_408e_4979_b4b4_d4b80316d7c2.slice/crio-0fb3a873b64dc04fbccd2650039ffcd5e1531e3533e4378a46754981ca0f7122 WatchSource:0}: Error finding container 0fb3a873b64dc04fbccd2650039ffcd5e1531e3533e4378a46754981ca0f7122: Status 404 returned error can't find the container with id 0fb3a873b64dc04fbccd2650039ffcd5e1531e3533e4378a46754981ca0f7122
Sep 07 10:31:24.813368 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:24.813332    2577 generic.go:356] "Generic (PLEG): container finished" podID="8bb4d11f-408e-4979-b4b4-d4b80316d7c2" containerID="7dc337778a88b521a69bed630ad6e2d8d6f15f7f09b9936300a7fc2c6f7a8b2b" exitCode=0
Sep 07 10:31:24.813534 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:24.813368    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t" event={"ID":"8bb4d11f-408e-4979-b4b4-d4b80316d7c2","Type":"ContainerDied","Data":"7dc337778a88b521a69bed630ad6e2d8d6f15f7f09b9936300a7fc2c6f7a8b2b"}
Sep 07 10:31:24.813534 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:24.813422    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t" event={"ID":"8bb4d11f-408e-4979-b4b4-d4b80316d7c2","Type":"ContainerStarted","Data":"0fb3a873b64dc04fbccd2650039ffcd5e1531e3533e4378a46754981ca0f7122"}
Sep 07 10:31:25.820292 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:25.820259    2577 generic.go:356] "Generic (PLEG): container finished" podID="8bb4d11f-408e-4979-b4b4-d4b80316d7c2" containerID="9e17c149cb5575b9703acf5f0aeb0f7bf6bb38a1410b8cb524961f74c3e6c574" exitCode=0
Sep 07 10:31:25.820785 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:25.820350    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t" event={"ID":"8bb4d11f-408e-4979-b4b4-d4b80316d7c2","Type":"ContainerDied","Data":"9e17c149cb5575b9703acf5f0aeb0f7bf6bb38a1410b8cb524961f74c3e6c574"}
Sep 07 10:31:26.827261 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:26.827228    2577 generic.go:356] "Generic (PLEG): container finished" podID="8bb4d11f-408e-4979-b4b4-d4b80316d7c2" containerID="f2421f4fbc3e59cbaf5b95c34e9ce987c268db70a8939842152c8607f817a363" exitCode=0
Sep 07 10:31:26.827662 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:26.827311    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t" event={"ID":"8bb4d11f-408e-4979-b4b4-d4b80316d7c2","Type":"ContainerDied","Data":"f2421f4fbc3e59cbaf5b95c34e9ce987c268db70a8939842152c8607f817a363"}
Sep 07 10:31:27.958074 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:27.958051    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"
Sep 07 10:31:28.054994 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:28.054955    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-util\") pod \"8bb4d11f-408e-4979-b4b4-d4b80316d7c2\" (UID: \"8bb4d11f-408e-4979-b4b4-d4b80316d7c2\") "
Sep 07 10:31:28.054994 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:28.054996    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-bundle\") pod \"8bb4d11f-408e-4979-b4b4-d4b80316d7c2\" (UID: \"8bb4d11f-408e-4979-b4b4-d4b80316d7c2\") "
Sep 07 10:31:28.055281 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:28.055063    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfjv5\" (UniqueName: \"kubernetes.io/projected/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-kube-api-access-xfjv5\") pod \"8bb4d11f-408e-4979-b4b4-d4b80316d7c2\" (UID: \"8bb4d11f-408e-4979-b4b4-d4b80316d7c2\") "
Sep 07 10:31:28.055919 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:28.055882    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-bundle" (OuterVolumeSpecName: "bundle") pod "8bb4d11f-408e-4979-b4b4-d4b80316d7c2" (UID: "8bb4d11f-408e-4979-b4b4-d4b80316d7c2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:31:28.057327 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:28.057298    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-kube-api-access-xfjv5" (OuterVolumeSpecName: "kube-api-access-xfjv5") pod "8bb4d11f-408e-4979-b4b4-d4b80316d7c2" (UID: "8bb4d11f-408e-4979-b4b4-d4b80316d7c2"). InnerVolumeSpecName "kube-api-access-xfjv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:31:28.060633 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:28.060609    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-util" (OuterVolumeSpecName: "util") pod "8bb4d11f-408e-4979-b4b4-d4b80316d7c2" (UID: "8bb4d11f-408e-4979-b4b4-d4b80316d7c2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:31:28.156515 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:28.156419    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-util\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:31:28.156515 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:28.156454    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-bundle\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:31:28.156515 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:28.156463    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfjv5\" (UniqueName: \"kubernetes.io/projected/8bb4d11f-408e-4979-b4b4-d4b80316d7c2-kube-api-access-xfjv5\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:31:28.837861 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:28.837826    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t" event={"ID":"8bb4d11f-408e-4979-b4b4-d4b80316d7c2","Type":"ContainerDied","Data":"0fb3a873b64dc04fbccd2650039ffcd5e1531e3533e4378a46754981ca0f7122"}
Sep 07 10:31:28.838002 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:28.837870    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0fb3a873b64dc04fbccd2650039ffcd5e1531e3533e4378a46754981ca0f7122"
Sep 07 10:31:28.838002 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:28.837869    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhxk8t"
Sep 07 10:31:34.473749 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.473711    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"]
Sep 07 10:31:34.474119 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.473996    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb4d11f-408e-4979-b4b4-d4b80316d7c2" containerName="util"
Sep 07 10:31:34.474119 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.474006    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb4d11f-408e-4979-b4b4-d4b80316d7c2" containerName="util"
Sep 07 10:31:34.474119 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.474020    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb4d11f-408e-4979-b4b4-d4b80316d7c2" containerName="pull"
Sep 07 10:31:34.474119 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.474025    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb4d11f-408e-4979-b4b4-d4b80316d7c2" containerName="pull"
Sep 07 10:31:34.474119 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.474031    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb4d11f-408e-4979-b4b4-d4b80316d7c2" containerName="extract"
Sep 07 10:31:34.474119 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.474037    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb4d11f-408e-4979-b4b4-d4b80316d7c2" containerName="extract"
Sep 07 10:31:34.474119 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.474092    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb4d11f-408e-4979-b4b4-d4b80316d7c2" containerName="extract"
Sep 07 10:31:34.478709 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.478687    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"
Sep 07 10:31:34.481701 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.481678    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9cdh5\""
Sep 07 10:31:34.481831 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.481811    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 07 10:31:34.482812 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.482794    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 07 10:31:34.486510 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.486486    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"]
Sep 07 10:31:34.608995 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.608951    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtxrs\" (UniqueName: \"kubernetes.io/projected/6afb77c0-5081-460a-ad81-55e8b24bfe9d-kube-api-access-mtxrs\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv\" (UID: \"6afb77c0-5081-460a-ad81-55e8b24bfe9d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"
Sep 07 10:31:34.608995 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.608999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6afb77c0-5081-460a-ad81-55e8b24bfe9d-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv\" (UID: \"6afb77c0-5081-460a-ad81-55e8b24bfe9d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"
Sep 07 10:31:34.609251 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.609159    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6afb77c0-5081-460a-ad81-55e8b24bfe9d-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv\" (UID: \"6afb77c0-5081-460a-ad81-55e8b24bfe9d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"
Sep 07 10:31:34.709582 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.709544    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtxrs\" (UniqueName: \"kubernetes.io/projected/6afb77c0-5081-460a-ad81-55e8b24bfe9d-kube-api-access-mtxrs\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv\" (UID: \"6afb77c0-5081-460a-ad81-55e8b24bfe9d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"
Sep 07 10:31:34.709582 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.709586    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6afb77c0-5081-460a-ad81-55e8b24bfe9d-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv\" (UID: \"6afb77c0-5081-460a-ad81-55e8b24bfe9d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"
Sep 07 10:31:34.709832 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.709636    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6afb77c0-5081-460a-ad81-55e8b24bfe9d-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv\" (UID: \"6afb77c0-5081-460a-ad81-55e8b24bfe9d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"
Sep 07 10:31:34.709974 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.709957    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6afb77c0-5081-460a-ad81-55e8b24bfe9d-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv\" (UID: \"6afb77c0-5081-460a-ad81-55e8b24bfe9d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"
Sep 07 10:31:34.710028 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.709994    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6afb77c0-5081-460a-ad81-55e8b24bfe9d-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv\" (UID: \"6afb77c0-5081-460a-ad81-55e8b24bfe9d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"
Sep 07 10:31:34.721870 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.721836    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtxrs\" (UniqueName: \"kubernetes.io/projected/6afb77c0-5081-460a-ad81-55e8b24bfe9d-kube-api-access-mtxrs\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv\" (UID: \"6afb77c0-5081-460a-ad81-55e8b24bfe9d\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"
Sep 07 10:31:34.788654 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:34.788581    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"
Sep 07 10:31:35.061888 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:35.061816    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"]
Sep 07 10:31:35.064931 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:31:35.064904    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6afb77c0_5081_460a_ad81_55e8b24bfe9d.slice/crio-0dfbaf650076ade37d7ea103a6f957d2826b82027fbb4d1c23e0d41bdd6d93ad WatchSource:0}: Error finding container 0dfbaf650076ade37d7ea103a6f957d2826b82027fbb4d1c23e0d41bdd6d93ad: Status 404 returned error can't find the container with id 0dfbaf650076ade37d7ea103a6f957d2826b82027fbb4d1c23e0d41bdd6d93ad
Sep 07 10:31:35.226597 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:35.226558    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:35.326934 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:35.326834    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:35.784955 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:35.784924    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:35.872406 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:35.872375    2577 generic.go:356] "Generic (PLEG): container finished" podID="6afb77c0-5081-460a-ad81-55e8b24bfe9d" containerID="e6159d28c7a30d3e75057465069c0a408a26e6c9f3b88ae7fd55330dd9e82c4a" exitCode=0
Sep 07 10:31:35.872593 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:35.872467    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv" event={"ID":"6afb77c0-5081-460a-ad81-55e8b24bfe9d","Type":"ContainerDied","Data":"e6159d28c7a30d3e75057465069c0a408a26e6c9f3b88ae7fd55330dd9e82c4a"}
Sep 07 10:31:35.872593 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:35.872526    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv" event={"ID":"6afb77c0-5081-460a-ad81-55e8b24bfe9d","Type":"ContainerStarted","Data":"0dfbaf650076ade37d7ea103a6f957d2826b82027fbb4d1c23e0d41bdd6d93ad"}
Sep 07 10:31:36.014871 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.014788    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:36.118858 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.118825    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:36.137459 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.137429    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"]
Sep 07 10:31:36.141008 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.140990    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.143637 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.143600    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 07 10:31:36.143637 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.143599    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-t7dj9\""
Sep 07 10:31:36.143833 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.143639    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 07 10:31:36.143833 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.143651    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 07 10:31:36.143996 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.143983    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 07 10:31:36.168489 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.168461    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"]
Sep 07 10:31:36.221200 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.221166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/118dfdf6-aae7-4c07-ac0b-7f0e42d93870-tmp\") pod \"opendatahub-operator-controller-manager-7f5bfcc886-76h68\" (UID: \"118dfdf6-aae7-4c07-ac0b-7f0e42d93870\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.221200 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.221202    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/118dfdf6-aae7-4c07-ac0b-7f0e42d93870-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f5bfcc886-76h68\" (UID: \"118dfdf6-aae7-4c07-ac0b-7f0e42d93870\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.221406 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.221221    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-572zb\" (UniqueName: \"kubernetes.io/projected/118dfdf6-aae7-4c07-ac0b-7f0e42d93870-kube-api-access-572zb\") pod \"opendatahub-operator-controller-manager-7f5bfcc886-76h68\" (UID: \"118dfdf6-aae7-4c07-ac0b-7f0e42d93870\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.221406 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.221244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/118dfdf6-aae7-4c07-ac0b-7f0e42d93870-manifests\") pod \"opendatahub-operator-controller-manager-7f5bfcc886-76h68\" (UID: \"118dfdf6-aae7-4c07-ac0b-7f0e42d93870\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.221406 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.221332    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/118dfdf6-aae7-4c07-ac0b-7f0e42d93870-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f5bfcc886-76h68\" (UID: \"118dfdf6-aae7-4c07-ac0b-7f0e42d93870\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.322789 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.322691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/118dfdf6-aae7-4c07-ac0b-7f0e42d93870-tmp\") pod \"opendatahub-operator-controller-manager-7f5bfcc886-76h68\" (UID: \"118dfdf6-aae7-4c07-ac0b-7f0e42d93870\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.322789 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.322735    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/118dfdf6-aae7-4c07-ac0b-7f0e42d93870-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f5bfcc886-76h68\" (UID: \"118dfdf6-aae7-4c07-ac0b-7f0e42d93870\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.322789 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.322758    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-572zb\" (UniqueName: \"kubernetes.io/projected/118dfdf6-aae7-4c07-ac0b-7f0e42d93870-kube-api-access-572zb\") pod \"opendatahub-operator-controller-manager-7f5bfcc886-76h68\" (UID: \"118dfdf6-aae7-4c07-ac0b-7f0e42d93870\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.322789 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.322783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/118dfdf6-aae7-4c07-ac0b-7f0e42d93870-manifests\") pod \"opendatahub-operator-controller-manager-7f5bfcc886-76h68\" (UID: \"118dfdf6-aae7-4c07-ac0b-7f0e42d93870\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.323069 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.322816    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/118dfdf6-aae7-4c07-ac0b-7f0e42d93870-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f5bfcc886-76h68\" (UID: \"118dfdf6-aae7-4c07-ac0b-7f0e42d93870\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.323288 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.323265    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/118dfdf6-aae7-4c07-ac0b-7f0e42d93870-manifests\") pod \"opendatahub-operator-controller-manager-7f5bfcc886-76h68\" (UID: \"118dfdf6-aae7-4c07-ac0b-7f0e42d93870\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.325320 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.325289    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/118dfdf6-aae7-4c07-ac0b-7f0e42d93870-tmp\") pod \"opendatahub-operator-controller-manager-7f5bfcc886-76h68\" (UID: \"118dfdf6-aae7-4c07-ac0b-7f0e42d93870\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.325436 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.325419    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/118dfdf6-aae7-4c07-ac0b-7f0e42d93870-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f5bfcc886-76h68\" (UID: \"118dfdf6-aae7-4c07-ac0b-7f0e42d93870\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.325599 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.325578    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/118dfdf6-aae7-4c07-ac0b-7f0e42d93870-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f5bfcc886-76h68\" (UID: \"118dfdf6-aae7-4c07-ac0b-7f0e42d93870\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.332973 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.332933    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-572zb\" (UniqueName: \"kubernetes.io/projected/118dfdf6-aae7-4c07-ac0b-7f0e42d93870-kube-api-access-572zb\") pod \"opendatahub-operator-controller-manager-7f5bfcc886-76h68\" (UID: \"118dfdf6-aae7-4c07-ac0b-7f0e42d93870\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.384616 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.384583    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:36.450723 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.450683    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:36.486072 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.486047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:36.585798 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.585717    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:36.687539 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.687504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:36.748404 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.748371    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"]
Sep 07 10:31:36.751912 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:31:36.751887    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod118dfdf6_aae7_4c07_ac0b_7f0e42d93870.slice/crio-2cd413c58ff749568ec5aa50a920570750156b6cc1179d617d61d9faa3200b7f WatchSource:0}: Error finding container 2cd413c58ff749568ec5aa50a920570750156b6cc1179d617d61d9faa3200b7f: Status 404 returned error can't find the container with id 2cd413c58ff749568ec5aa50a920570750156b6cc1179d617d61d9faa3200b7f
Sep 07 10:31:36.787615 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.787578    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:36.878571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.878480    2577 generic.go:356] "Generic (PLEG): container finished" podID="6afb77c0-5081-460a-ad81-55e8b24bfe9d" containerID="2c1b51bd6defafdbdf2f1a136b76161e6b494c2622d234c7542c47d237e2803c" exitCode=0
Sep 07 10:31:36.878571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.878543    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv" event={"ID":"6afb77c0-5081-460a-ad81-55e8b24bfe9d","Type":"ContainerDied","Data":"2c1b51bd6defafdbdf2f1a136b76161e6b494c2622d234c7542c47d237e2803c"}
Sep 07 10:31:36.880068 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:36.880041    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68" event={"ID":"118dfdf6-aae7-4c07-ac0b-7f0e42d93870","Type":"ContainerStarted","Data":"2cd413c58ff749568ec5aa50a920570750156b6cc1179d617d61d9faa3200b7f"}
Sep 07 10:31:37.890188 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:37.889961    2577 generic.go:356] "Generic (PLEG): container finished" podID="6afb77c0-5081-460a-ad81-55e8b24bfe9d" containerID="7458e68fd695dab2b8e8383e2ae90a4f1eb36c74bdd5117e12945d4f1bb07b37" exitCode=0
Sep 07 10:31:37.890188 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:37.890102    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv" event={"ID":"6afb77c0-5081-460a-ad81-55e8b24bfe9d","Type":"ContainerDied","Data":"7458e68fd695dab2b8e8383e2ae90a4f1eb36c74bdd5117e12945d4f1bb07b37"}
Sep 07 10:31:39.899552 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:39.899519    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"
Sep 07 10:31:39.901231 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:39.901198    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv" event={"ID":"6afb77c0-5081-460a-ad81-55e8b24bfe9d","Type":"ContainerDied","Data":"0dfbaf650076ade37d7ea103a6f957d2826b82027fbb4d1c23e0d41bdd6d93ad"}
Sep 07 10:31:39.901347 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:39.901241    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0dfbaf650076ade37d7ea103a6f957d2826b82027fbb4d1c23e0d41bdd6d93ad"
Sep 07 10:31:40.059075 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:40.059042    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtxrs\" (UniqueName: \"kubernetes.io/projected/6afb77c0-5081-460a-ad81-55e8b24bfe9d-kube-api-access-mtxrs\") pod \"6afb77c0-5081-460a-ad81-55e8b24bfe9d\" (UID: \"6afb77c0-5081-460a-ad81-55e8b24bfe9d\") "
Sep 07 10:31:40.059281 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:40.059107    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6afb77c0-5081-460a-ad81-55e8b24bfe9d-bundle\") pod \"6afb77c0-5081-460a-ad81-55e8b24bfe9d\" (UID: \"6afb77c0-5081-460a-ad81-55e8b24bfe9d\") "
Sep 07 10:31:40.059281 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:40.059126    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6afb77c0-5081-460a-ad81-55e8b24bfe9d-util\") pod \"6afb77c0-5081-460a-ad81-55e8b24bfe9d\" (UID: \"6afb77c0-5081-460a-ad81-55e8b24bfe9d\") "
Sep 07 10:31:40.059775 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:40.059748    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6afb77c0-5081-460a-ad81-55e8b24bfe9d-bundle" (OuterVolumeSpecName: "bundle") pod "6afb77c0-5081-460a-ad81-55e8b24bfe9d" (UID: "6afb77c0-5081-460a-ad81-55e8b24bfe9d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:31:40.061458 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:40.061428    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6afb77c0-5081-460a-ad81-55e8b24bfe9d-kube-api-access-mtxrs" (OuterVolumeSpecName: "kube-api-access-mtxrs") pod "6afb77c0-5081-460a-ad81-55e8b24bfe9d" (UID: "6afb77c0-5081-460a-ad81-55e8b24bfe9d"). InnerVolumeSpecName "kube-api-access-mtxrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:31:40.064602 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:40.064576    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6afb77c0-5081-460a-ad81-55e8b24bfe9d-util" (OuterVolumeSpecName: "util") pod "6afb77c0-5081-460a-ad81-55e8b24bfe9d" (UID: "6afb77c0-5081-460a-ad81-55e8b24bfe9d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:31:40.160660 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:40.160565    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtxrs\" (UniqueName: \"kubernetes.io/projected/6afb77c0-5081-460a-ad81-55e8b24bfe9d-kube-api-access-mtxrs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:31:40.160660 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:40.160598    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6afb77c0-5081-460a-ad81-55e8b24bfe9d-bundle\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:31:40.160660 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:40.160607    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6afb77c0-5081-460a-ad81-55e8b24bfe9d-util\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:31:40.907299 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:40.907262    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68" event={"ID":"118dfdf6-aae7-4c07-ac0b-7f0e42d93870","Type":"ContainerStarted","Data":"ac5d0a45687e64751dc317d60f061acca5cbeb1fb19327313a378c5f96d3da13"}
Sep 07 10:31:40.907299 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:40.907288    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0sqqbv"
Sep 07 10:31:41.913543 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:41.913508    2577 generic.go:356] "Generic (PLEG): container finished" podID="118dfdf6-aae7-4c07-ac0b-7f0e42d93870" containerID="ac5d0a45687e64751dc317d60f061acca5cbeb1fb19327313a378c5f96d3da13" exitCode=0
Sep 07 10:31:41.913918 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:41.913598    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68" event={"ID":"118dfdf6-aae7-4c07-ac0b-7f0e42d93870","Type":"ContainerDied","Data":"ac5d0a45687e64751dc317d60f061acca5cbeb1fb19327313a378c5f96d3da13"}
Sep 07 10:31:42.920043 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:42.920005    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68" event={"ID":"118dfdf6-aae7-4c07-ac0b-7f0e42d93870","Type":"ContainerStarted","Data":"8f95d2243214bc4f1d1fab9539f04b2c2604f42bbc90fc1621785cb67449311c"}
Sep 07 10:31:42.920739 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:42.920177    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:42.947252 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:42.946655    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68" podStartSLOduration=2.188450943 podStartE2EDuration="6.946631514s" totalImagesPullingTime="3.7826162s" totalInitContainerRuntime="975.564371ms" isStatefulPod=false podCreationTimestamp="2026-09-07 10:31:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:31:42.944555416 +0000 UTC m=+706.678729654" watchObservedRunningTime="2026-09-07 10:31:42.946631514 +0000 UTC m=+706.680805749"
Sep 07 10:31:43.688872 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:43.688841    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:47.167963 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:47.167914    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:47.287610 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:47.287573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:53.927153 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:53.927051    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7f5bfcc886-76h68"
Sep 07 10:31:55.407277 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.407242    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:55.541546 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.541508    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:55.833548 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.833516    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7794764676-62lq5"]
Sep 07 10:31:55.836277 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.834374    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6afb77c0-5081-460a-ad81-55e8b24bfe9d" containerName="extract"
Sep 07 10:31:55.836277 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.834407    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6afb77c0-5081-460a-ad81-55e8b24bfe9d" containerName="extract"
Sep 07 10:31:55.836277 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.834468    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6afb77c0-5081-460a-ad81-55e8b24bfe9d" containerName="util"
Sep 07 10:31:55.836277 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.834477    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6afb77c0-5081-460a-ad81-55e8b24bfe9d" containerName="util"
Sep 07 10:31:55.836277 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.834504    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6afb77c0-5081-460a-ad81-55e8b24bfe9d" containerName="pull"
Sep 07 10:31:55.836277 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.834512    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6afb77c0-5081-460a-ad81-55e8b24bfe9d" containerName="pull"
Sep 07 10:31:55.836277 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.834704    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6afb77c0-5081-460a-ad81-55e8b24bfe9d" containerName="extract"
Sep 07 10:31:55.838562 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.838533    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7794764676-62lq5"
Sep 07 10:31:55.842346 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.842321    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 07 10:31:55.842477 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.842321    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 07 10:31:55.853354 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.853324    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:55.865490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.865468    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7794764676-62lq5"]
Sep 07 10:31:55.964534 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.964505    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:55.991682 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.991658    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67fd58dd-c355-470a-8643-6dc31ef2ae9f-tmp\") pod \"kube-auth-proxy-7794764676-62lq5\" (UID: \"67fd58dd-c355-470a-8643-6dc31ef2ae9f\") " pod="openshift-ingress/kube-auth-proxy-7794764676-62lq5"
Sep 07 10:31:55.991822 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.991698    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgl4g\" (UniqueName: \"kubernetes.io/projected/67fd58dd-c355-470a-8643-6dc31ef2ae9f-kube-api-access-jgl4g\") pod \"kube-auth-proxy-7794764676-62lq5\" (UID: \"67fd58dd-c355-470a-8643-6dc31ef2ae9f\") " pod="openshift-ingress/kube-auth-proxy-7794764676-62lq5"
Sep 07 10:31:55.991822 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:55.991780    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67fd58dd-c355-470a-8643-6dc31ef2ae9f-tls-certs\") pod \"kube-auth-proxy-7794764676-62lq5\" (UID: \"67fd58dd-c355-470a-8643-6dc31ef2ae9f\") " pod="openshift-ingress/kube-auth-proxy-7794764676-62lq5"
Sep 07 10:31:56.093025 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:56.092942    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgl4g\" (UniqueName: \"kubernetes.io/projected/67fd58dd-c355-470a-8643-6dc31ef2ae9f-kube-api-access-jgl4g\") pod \"kube-auth-proxy-7794764676-62lq5\" (UID: \"67fd58dd-c355-470a-8643-6dc31ef2ae9f\") " pod="openshift-ingress/kube-auth-proxy-7794764676-62lq5"
Sep 07 10:31:56.093025 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:56.093008    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67fd58dd-c355-470a-8643-6dc31ef2ae9f-tls-certs\") pod \"kube-auth-proxy-7794764676-62lq5\" (UID: \"67fd58dd-c355-470a-8643-6dc31ef2ae9f\") " pod="openshift-ingress/kube-auth-proxy-7794764676-62lq5"
Sep 07 10:31:56.093252 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:56.093047    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67fd58dd-c355-470a-8643-6dc31ef2ae9f-tmp\") pod \"kube-auth-proxy-7794764676-62lq5\" (UID: \"67fd58dd-c355-470a-8643-6dc31ef2ae9f\") " pod="openshift-ingress/kube-auth-proxy-7794764676-62lq5"
Sep 07 10:31:56.095309 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:56.095276    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67fd58dd-c355-470a-8643-6dc31ef2ae9f-tmp\") pod \"kube-auth-proxy-7794764676-62lq5\" (UID: \"67fd58dd-c355-470a-8643-6dc31ef2ae9f\") " pod="openshift-ingress/kube-auth-proxy-7794764676-62lq5"
Sep 07 10:31:56.095548 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:56.095531    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67fd58dd-c355-470a-8643-6dc31ef2ae9f-tls-certs\") pod \"kube-auth-proxy-7794764676-62lq5\" (UID: \"67fd58dd-c355-470a-8643-6dc31ef2ae9f\") " pod="openshift-ingress/kube-auth-proxy-7794764676-62lq5"
Sep 07 10:31:56.104987 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:56.104962    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgl4g\" (UniqueName: \"kubernetes.io/projected/67fd58dd-c355-470a-8643-6dc31ef2ae9f-kube-api-access-jgl4g\") pod \"kube-auth-proxy-7794764676-62lq5\" (UID: \"67fd58dd-c355-470a-8643-6dc31ef2ae9f\") " pod="openshift-ingress/kube-auth-proxy-7794764676-62lq5"
Sep 07 10:31:56.148662 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:56.148637    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7794764676-62lq5"
Sep 07 10:31:56.434530 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:56.434502    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7794764676-62lq5"]
Sep 07 10:31:56.436508 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:31:56.436475    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67fd58dd_c355_470a_8643_6dc31ef2ae9f.slice/crio-536f8afefa186035195d23730a0d8956d29cb8662814f5af8a175e8a9f5d5d62 WatchSource:0}: Error finding container 536f8afefa186035195d23730a0d8956d29cb8662814f5af8a175e8a9f5d5d62: Status 404 returned error can't find the container with id 536f8afefa186035195d23730a0d8956d29cb8662814f5af8a175e8a9f5d5d62
Sep 07 10:31:56.983147 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:56.983097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7794764676-62lq5" event={"ID":"67fd58dd-c355-470a-8643-6dc31ef2ae9f","Type":"ContainerStarted","Data":"536f8afefa186035195d23730a0d8956d29cb8662814f5af8a175e8a9f5d5d62"}
Sep 07 10:31:57.355238 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.355153    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:57.377793 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.377766    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"]
Sep 07 10:31:57.381259 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.381219    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.387899 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.387559    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 07 10:31:57.387899 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.387793    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 07 10:31:57.387899 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.387842    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 07 10:31:57.388284 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.388173    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2pfb6\""
Sep 07 10:31:57.390061 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.390039    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 07 10:31:57.475123 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.475044    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"]
Sep 07 10:31:57.503243 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.503212    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.503420 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.503255    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d06a2864-ee88-4580-b3e8-ad23bae0cd33-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.503420 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.503285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.503420 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.503348    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzcsk\" (UniqueName: \"kubernetes.io/projected/d06a2864-ee88-4580-b3e8-ad23bae0cd33-kube-api-access-bzcsk\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.503582 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.503446    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.503582 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.503518    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.503582 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.503564    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.524046 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.524008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:57.604038 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.604002    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.604230 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.604065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.604230 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.604114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.604230 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.604171    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.604230 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.604211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d06a2864-ee88-4580-b3e8-ad23bae0cd33-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.604450 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.604238    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.604450 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.604267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzcsk\" (UniqueName: \"kubernetes.io/projected/d06a2864-ee88-4580-b3e8-ad23bae0cd33-kube-api-access-bzcsk\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.604894 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.604748    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.607383 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.607319    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.607501 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.607443    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d06a2864-ee88-4580-b3e8-ad23bae0cd33-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.607625 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.607595    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.607702 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.607681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.619885 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.619850    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzcsk\" (UniqueName: \"kubernetes.io/projected/d06a2864-ee88-4580-b3e8-ad23bae0cd33-kube-api-access-bzcsk\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.623068 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.623006    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-qtbcp\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.695170 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.695082    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:31:57.984925 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:57.984891    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"]
Sep 07 10:31:58.658762 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:58.658726    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:31:59.130046 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:31:59.130003    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd06a2864_ee88_4580_b3e8_ad23bae0cd33.slice/crio-34436eb8d3943adcebed3aad834a8ef12d16428176517015423421138d4c4f00 WatchSource:0}: Error finding container 34436eb8d3943adcebed3aad834a8ef12d16428176517015423421138d4c4f00: Status 404 returned error can't find the container with id 34436eb8d3943adcebed3aad834a8ef12d16428176517015423421138d4c4f00
Sep 07 10:31:59.192069 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:31:59.192049    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 07 10:32:00.003516 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:00.003458    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7794764676-62lq5" event={"ID":"67fd58dd-c355-470a-8643-6dc31ef2ae9f","Type":"ContainerStarted","Data":"5c1977a85c0e778bbcc2b78402b99810d71895aa3b8769d35b113e420b485b60"}
Sep 07 10:32:00.006196 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:00.006160    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp" event={"ID":"d06a2864-ee88-4580-b3e8-ad23bae0cd33","Type":"ContainerStarted","Data":"34436eb8d3943adcebed3aad834a8ef12d16428176517015423421138d4c4f00"}
Sep 07 10:32:00.022933 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:00.022877    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7794764676-62lq5" podStartSLOduration=2.271816098 podStartE2EDuration="5.022860383s" totalImagesPullingTime="2.751044285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:31:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:32:00.021346857 +0000 UTC m=+723.755521091" watchObservedRunningTime="2026-09-07 10:32:00.022860383 +0000 UTC m=+723.757034616"
Sep 07 10:32:01.620305 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:01.620262    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 07 10:32:01.620570 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:01.620345    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 07 10:32:02.019246 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:02.019209    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp" event={"ID":"d06a2864-ee88-4580-b3e8-ad23bae0cd33","Type":"ContainerStarted","Data":"48ecb536e3a10df57c7c3d52fb30986de33d1fcb2a96c6900fdb96a57af45dba"}
Sep 07 10:32:02.019448 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:02.019331    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:32:02.021148 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:02.021108    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-qtbcp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 07 10:32:02.021286 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:02.021187    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp" podUID="d06a2864-ee88-4580-b3e8-ad23bae0cd33" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 07 10:32:02.043411 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:02.043361    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp" podStartSLOduration=2.5560102000000002 podStartE2EDuration="5.043343695s" totalImagesPullingTime="2.487333495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:31:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:32:02.040295833 +0000 UTC m=+725.774470080" watchObservedRunningTime="2026-09-07 10:32:02.043343695 +0000 UTC m=+725.777517927"
Sep 07 10:32:02.310368 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:02.310258    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:02.409262 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:02.409234    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:03.024158 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:03.024111    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:32:03.088195 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:03.088121    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:03.298559 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:03.298462    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:03.453231 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:03.453197    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:06.592720 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:06.592687    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:06.691258 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:06.691226    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:06.790769 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:06.790739    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:07.058739 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:07.058706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:07.155686 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:07.155655    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:07.253393 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:07.253355    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:16.050983 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.050946    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:16.116489 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.116450    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q"]
Sep 07 10:32:16.126439 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.126415    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q"
Sep 07 10:32:16.131703 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.131661    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 07 10:32:16.131926 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.131911    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-9pcfg\""
Sep 07 10:32:16.133681 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.133660    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj"]
Sep 07 10:32:16.136857 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.136839    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj"
Sep 07 10:32:16.139961 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.139939    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 07 10:32:16.144685 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.144665    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q"]
Sep 07 10:32:16.145104 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.145087    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-bcqn5\""
Sep 07 10:32:16.149845 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.149822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfms7\" (UniqueName: \"kubernetes.io/projected/43f23687-6876-4cb7-9b5f-856de753f721-kube-api-access-vfms7\") pod \"llmisvc-controller-manager-777c6bd7d5-sj2nj\" (UID: \"43f23687-6876-4cb7-9b5f-856de753f721\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj"
Sep 07 10:32:16.149923 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.149867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43f23687-6876-4cb7-9b5f-856de753f721-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sj2nj\" (UID: \"43f23687-6876-4cb7-9b5f-856de753f721\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj"
Sep 07 10:32:16.149966 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.149940    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwbpp\" (UniqueName: \"kubernetes.io/projected/462b889c-0701-47da-a0df-53ff3a1cc19d-kube-api-access-mwbpp\") pod \"kserve-controller-manager-d6f4b65b8-gfk2q\" (UID: \"462b889c-0701-47da-a0df-53ff3a1cc19d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q"
Sep 07 10:32:16.150000 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.149981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/462b889c-0701-47da-a0df-53ff3a1cc19d-cert\") pod \"kserve-controller-manager-d6f4b65b8-gfk2q\" (UID: \"462b889c-0701-47da-a0df-53ff3a1cc19d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q"
Sep 07 10:32:16.153310 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.153290    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj"]
Sep 07 10:32:16.164200 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.164172    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:16.250640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.250614    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfms7\" (UniqueName: \"kubernetes.io/projected/43f23687-6876-4cb7-9b5f-856de753f721-kube-api-access-vfms7\") pod \"llmisvc-controller-manager-777c6bd7d5-sj2nj\" (UID: \"43f23687-6876-4cb7-9b5f-856de753f721\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj"
Sep 07 10:32:16.250760 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.250648    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43f23687-6876-4cb7-9b5f-856de753f721-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sj2nj\" (UID: \"43f23687-6876-4cb7-9b5f-856de753f721\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj"
Sep 07 10:32:16.250760 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.250676    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwbpp\" (UniqueName: \"kubernetes.io/projected/462b889c-0701-47da-a0df-53ff3a1cc19d-kube-api-access-mwbpp\") pod \"kserve-controller-manager-d6f4b65b8-gfk2q\" (UID: \"462b889c-0701-47da-a0df-53ff3a1cc19d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q"
Sep 07 10:32:16.250851 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:32:16.250769    2577 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 07 10:32:16.250851 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:32:16.250825    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43f23687-6876-4cb7-9b5f-856de753f721-cert podName:43f23687-6876-4cb7-9b5f-856de753f721 nodeName:}" failed. No retries permitted until 2026-09-07 10:32:16.750810697 +0000 UTC m=+740.484984908 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/43f23687-6876-4cb7-9b5f-856de753f721-cert") pod "llmisvc-controller-manager-777c6bd7d5-sj2nj" (UID: "43f23687-6876-4cb7-9b5f-856de753f721") : secret "llmisvc-webhook-server-cert" not found
Sep 07 10:32:16.250957 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.250879    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/462b889c-0701-47da-a0df-53ff3a1cc19d-cert\") pod \"kserve-controller-manager-d6f4b65b8-gfk2q\" (UID: \"462b889c-0701-47da-a0df-53ff3a1cc19d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q"
Sep 07 10:32:16.250995 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:32:16.250987    2577 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 07 10:32:16.251048 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:32:16.251037    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/462b889c-0701-47da-a0df-53ff3a1cc19d-cert podName:462b889c-0701-47da-a0df-53ff3a1cc19d nodeName:}" failed. No retries permitted until 2026-09-07 10:32:16.751019506 +0000 UTC m=+740.485193718 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/462b889c-0701-47da-a0df-53ff3a1cc19d-cert") pod "kserve-controller-manager-d6f4b65b8-gfk2q" (UID: "462b889c-0701-47da-a0df-53ff3a1cc19d") : secret "kserve-webhook-server-cert" not found
Sep 07 10:32:16.260022 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.259991    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwbpp\" (UniqueName: \"kubernetes.io/projected/462b889c-0701-47da-a0df-53ff3a1cc19d-kube-api-access-mwbpp\") pod \"kserve-controller-manager-d6f4b65b8-gfk2q\" (UID: \"462b889c-0701-47da-a0df-53ff3a1cc19d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q"
Sep 07 10:32:16.260476 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.260458    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfms7\" (UniqueName: \"kubernetes.io/projected/43f23687-6876-4cb7-9b5f-856de753f721-kube-api-access-vfms7\") pod \"llmisvc-controller-manager-777c6bd7d5-sj2nj\" (UID: \"43f23687-6876-4cb7-9b5f-856de753f721\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj"
Sep 07 10:32:16.521718 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.521685    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:16.665341 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.665311    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:16.753671 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.753644    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/462b889c-0701-47da-a0df-53ff3a1cc19d-cert\") pod \"kserve-controller-manager-d6f4b65b8-gfk2q\" (UID: \"462b889c-0701-47da-a0df-53ff3a1cc19d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q"
Sep 07 10:32:16.753814 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.753699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43f23687-6876-4cb7-9b5f-856de753f721-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sj2nj\" (UID: \"43f23687-6876-4cb7-9b5f-856de753f721\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj"
Sep 07 10:32:16.756246 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.756222    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/462b889c-0701-47da-a0df-53ff3a1cc19d-cert\") pod \"kserve-controller-manager-d6f4b65b8-gfk2q\" (UID: \"462b889c-0701-47da-a0df-53ff3a1cc19d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q"
Sep 07 10:32:16.756360 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:16.756344    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43f23687-6876-4cb7-9b5f-856de753f721-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sj2nj\" (UID: \"43f23687-6876-4cb7-9b5f-856de753f721\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj"
Sep 07 10:32:17.036947 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:17.036920    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q"
Sep 07 10:32:17.046204 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:17.046182    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj"
Sep 07 10:32:17.323181 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:17.323151    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q"]
Sep 07 10:32:17.324585 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:32:17.324562    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod462b889c_0701_47da_a0df_53ff3a1cc19d.slice/crio-19a759d0b54df6d35a511e92091cab89c8246b973e42b2ae7b1b72b9f4f9284e WatchSource:0}: Error finding container 19a759d0b54df6d35a511e92091cab89c8246b973e42b2ae7b1b72b9f4f9284e: Status 404 returned error can't find the container with id 19a759d0b54df6d35a511e92091cab89c8246b973e42b2ae7b1b72b9f4f9284e
Sep 07 10:32:17.344657 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:17.344635    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj"]
Sep 07 10:32:17.345104 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:32:17.345082    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod43f23687_6876_4cb7_9b5f_856de753f721.slice/crio-2c4f498274061fb3962505f0362a4eccd38d6d97294406efd15d9b4701a570cd WatchSource:0}: Error finding container 2c4f498274061fb3962505f0362a4eccd38d6d97294406efd15d9b4701a570cd: Status 404 returned error can't find the container with id 2c4f498274061fb3962505f0362a4eccd38d6d97294406efd15d9b4701a570cd
Sep 07 10:32:17.361213 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:17.361189    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:17.458040 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:17.458006    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:17.568328 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:17.568290    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:32:18.003152 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.003102    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"]
Sep 07 10:32:18.003364 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.003340    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp" podUID="d06a2864-ee88-4580-b3e8-ad23bae0cd33" containerName="discovery" containerID="cri-o://48ecb536e3a10df57c7c3d52fb30986de33d1fcb2a96c6900fdb96a57af45dba" gracePeriod=30
Sep 07 10:32:18.024355 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.024323    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-qtbcp container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.35:8080/ready\": dial tcp 10.132.0.35:8080: connect: connection refused" start-of-body=
Sep 07 10:32:18.024477 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.024383    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp" podUID="d06a2864-ee88-4580-b3e8-ad23bae0cd33" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.35:8080/ready\": dial tcp 10.132.0.35:8080: connect: connection refused"
Sep 07 10:32:18.101852 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.101813    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj" event={"ID":"43f23687-6876-4cb7-9b5f-856de753f721","Type":"ContainerStarted","Data":"2c4f498274061fb3962505f0362a4eccd38d6d97294406efd15d9b4701a570cd"}
Sep 07 10:32:18.103328 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.103300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q" event={"ID":"462b889c-0701-47da-a0df-53ff3a1cc19d","Type":"ContainerStarted","Data":"19a759d0b54df6d35a511e92091cab89c8246b973e42b2ae7b1b72b9f4f9284e"}
Sep 07 10:32:18.394037 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.394004    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:32:18.469203 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.468873    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-csr-ca-configmap\") pod \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") "
Sep 07 10:32:18.469203 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.468935    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-cacerts\") pod \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") "
Sep 07 10:32:18.469203 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.468986    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzcsk\" (UniqueName: \"kubernetes.io/projected/d06a2864-ee88-4580-b3e8-ad23bae0cd33-kube-api-access-bzcsk\") pod \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") "
Sep 07 10:32:18.469203 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.469041    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-token\") pod \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") "
Sep 07 10:32:18.469203 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.469091    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-kubeconfig\") pod \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") "
Sep 07 10:32:18.469203 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.469118    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-csr-dns-cert\") pod \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") "
Sep 07 10:32:18.470286 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.469672    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d06a2864-ee88-4580-b3e8-ad23bae0cd33-local-certs\") pod \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\" (UID: \"d06a2864-ee88-4580-b3e8-ad23bae0cd33\") "
Sep 07 10:32:18.470286 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.469969    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "d06a2864-ee88-4580-b3e8-ad23bae0cd33" (UID: "d06a2864-ee88-4580-b3e8-ad23bae0cd33"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:32:18.472942 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.472918    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-token" (OuterVolumeSpecName: "istio-token") pod "d06a2864-ee88-4580-b3e8-ad23bae0cd33" (UID: "d06a2864-ee88-4580-b3e8-ad23bae0cd33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:32:18.473035 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.473022    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "d06a2864-ee88-4580-b3e8-ad23bae0cd33" (UID: "d06a2864-ee88-4580-b3e8-ad23bae0cd33"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:32:18.474791 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.474745    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "d06a2864-ee88-4580-b3e8-ad23bae0cd33" (UID: "d06a2864-ee88-4580-b3e8-ad23bae0cd33"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:32:18.476119 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.475368    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d06a2864-ee88-4580-b3e8-ad23bae0cd33-kube-api-access-bzcsk" (OuterVolumeSpecName: "kube-api-access-bzcsk") pod "d06a2864-ee88-4580-b3e8-ad23bae0cd33" (UID: "d06a2864-ee88-4580-b3e8-ad23bae0cd33"). InnerVolumeSpecName "kube-api-access-bzcsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:32:18.476348 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.476321    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-cacerts" (OuterVolumeSpecName: "cacerts") pod "d06a2864-ee88-4580-b3e8-ad23bae0cd33" (UID: "d06a2864-ee88-4580-b3e8-ad23bae0cd33"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:32:18.476348 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.476338    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d06a2864-ee88-4580-b3e8-ad23bae0cd33-local-certs" (OuterVolumeSpecName: "local-certs") pod "d06a2864-ee88-4580-b3e8-ad23bae0cd33" (UID: "d06a2864-ee88-4580-b3e8-ad23bae0cd33"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:32:18.571644 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.571559    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzcsk\" (UniqueName: \"kubernetes.io/projected/d06a2864-ee88-4580-b3e8-ad23bae0cd33-kube-api-access-bzcsk\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:32:18.571644 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.571598    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-token\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:32:18.571644 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.571613    2577 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-kubeconfig\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:32:18.571644 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.571625    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-csr-dns-cert\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:32:18.571644 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.571638    2577 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d06a2864-ee88-4580-b3e8-ad23bae0cd33-local-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:32:18.572004 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.571653    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d06a2864-ee88-4580-b3e8-ad23bae0cd33-istio-csr-ca-configmap\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:32:18.572004 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:18.571664    2577 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d06a2864-ee88-4580-b3e8-ad23bae0cd33-cacerts\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:32:19.111460 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:19.111425    2577 generic.go:356] "Generic (PLEG): container finished" podID="d06a2864-ee88-4580-b3e8-ad23bae0cd33" containerID="48ecb536e3a10df57c7c3d52fb30986de33d1fcb2a96c6900fdb96a57af45dba" exitCode=0
Sep 07 10:32:19.111643 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:19.111478    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"
Sep 07 10:32:19.111643 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:19.111491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp" event={"ID":"d06a2864-ee88-4580-b3e8-ad23bae0cd33","Type":"ContainerDied","Data":"48ecb536e3a10df57c7c3d52fb30986de33d1fcb2a96c6900fdb96a57af45dba"}
Sep 07 10:32:19.111643 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:19.111538    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp" event={"ID":"d06a2864-ee88-4580-b3e8-ad23bae0cd33","Type":"ContainerDied","Data":"34436eb8d3943adcebed3aad834a8ef12d16428176517015423421138d4c4f00"}
Sep 07 10:32:19.111643 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:19.111567    2577 scope.go:117] "RemoveContainer" containerID="48ecb536e3a10df57c7c3d52fb30986de33d1fcb2a96c6900fdb96a57af45dba"
Sep 07 10:32:19.131791 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:19.131765    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"]
Sep 07 10:32:19.136269 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:19.136247    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-qtbcp"]
Sep 07 10:32:20.112629 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:20.112436    2577 scope.go:117] "RemoveContainer" containerID="48ecb536e3a10df57c7c3d52fb30986de33d1fcb2a96c6900fdb96a57af45dba"
Sep 07 10:32:20.112841 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:32:20.112774    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48ecb536e3a10df57c7c3d52fb30986de33d1fcb2a96c6900fdb96a57af45dba\": container with ID starting with 48ecb536e3a10df57c7c3d52fb30986de33d1fcb2a96c6900fdb96a57af45dba not found: ID does not exist" containerID="48ecb536e3a10df57c7c3d52fb30986de33d1fcb2a96c6900fdb96a57af45dba"
Sep 07 10:32:20.112841 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:20.112807    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48ecb536e3a10df57c7c3d52fb30986de33d1fcb2a96c6900fdb96a57af45dba"} err="failed to get container status \"48ecb536e3a10df57c7c3d52fb30986de33d1fcb2a96c6900fdb96a57af45dba\": rpc error: code = NotFound desc = could not find container \"48ecb536e3a10df57c7c3d52fb30986de33d1fcb2a96c6900fdb96a57af45dba\": container with ID starting with 48ecb536e3a10df57c7c3d52fb30986de33d1fcb2a96c6900fdb96a57af45dba not found: ID does not exist"
Sep 07 10:32:20.784798 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:20.784759    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d06a2864-ee88-4580-b3e8-ad23bae0cd33" path="/var/lib/kubelet/pods/d06a2864-ee88-4580-b3e8-ad23bae0cd33/volumes"
Sep 07 10:32:21.124245 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:21.124161    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj" event={"ID":"43f23687-6876-4cb7-9b5f-856de753f721","Type":"ContainerStarted","Data":"8aa9839f87c1d9c972149d609ca34563653f004e0d564c4ce327c196b09fede0"}
Sep 07 10:32:21.124620 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:21.124244    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj"
Sep 07 10:32:21.125852 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:21.125828    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q" event={"ID":"462b889c-0701-47da-a0df-53ff3a1cc19d","Type":"ContainerStarted","Data":"b8171a0cb9db882dbac2a944fe3a88d7cadab32a2154a004a5046b6c911ed4a7"}
Sep 07 10:32:21.125963 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:21.125947    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q"
Sep 07 10:32:21.166173 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:21.166110    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj" podStartSLOduration=2.38693236 podStartE2EDuration="5.166096622s" totalImagesPullingTime="2.779164262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:32:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:32:21.16394069 +0000 UTC m=+744.898114922" watchObservedRunningTime="2026-09-07 10:32:21.166096622 +0000 UTC m=+744.900270857"
Sep 07 10:32:21.203870 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:21.203825    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q" podStartSLOduration=2.408786538 podStartE2EDuration="5.203811733s" totalImagesPullingTime="2.795025195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:32:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:32:21.203651532 +0000 UTC m=+744.937825765" watchObservedRunningTime="2026-09-07 10:32:21.203811733 +0000 UTC m=+744.937985965"
Sep 07 10:32:52.132550 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:52.132521    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sj2nj"
Sep 07 10:32:52.135576 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:32:52.135556    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gfk2q"
Sep 07 10:33:07.524925 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:07.524878    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:07.620759 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:07.620734    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:11.042835 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.042805    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"]
Sep 07 10:33:11.043299 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.043154    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d06a2864-ee88-4580-b3e8-ad23bae0cd33" containerName="discovery"
Sep 07 10:33:11.043299 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.043166    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d06a2864-ee88-4580-b3e8-ad23bae0cd33" containerName="discovery"
Sep 07 10:33:11.043299 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.043230    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d06a2864-ee88-4580-b3e8-ad23bae0cd33" containerName="discovery"
Sep 07 10:33:11.048171 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.048153    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"
Sep 07 10:33:11.050785 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.050765    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 07 10:33:11.050888 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.050867    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 07 10:33:11.051961 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.051939    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-9cdh5\""
Sep 07 10:33:11.054542 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.054522    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"]
Sep 07 10:33:11.098048 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.098025    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs6rn\" (UniqueName: \"kubernetes.io/projected/347a269b-3cf1-4b8c-8835-7773feaa7945-kube-api-access-hs6rn\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5\" (UID: \"347a269b-3cf1-4b8c-8835-7773feaa7945\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"
Sep 07 10:33:11.098186 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.098080    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/347a269b-3cf1-4b8c-8835-7773feaa7945-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5\" (UID: \"347a269b-3cf1-4b8c-8835-7773feaa7945\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"
Sep 07 10:33:11.098186 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.098161    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/347a269b-3cf1-4b8c-8835-7773feaa7945-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5\" (UID: \"347a269b-3cf1-4b8c-8835-7773feaa7945\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"
Sep 07 10:33:11.140757 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.140737    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"]
Sep 07 10:33:11.144004 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.143990    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"
Sep 07 10:33:11.153415 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.153392    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"]
Sep 07 10:33:11.198925 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.198905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/347a269b-3cf1-4b8c-8835-7773feaa7945-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5\" (UID: \"347a269b-3cf1-4b8c-8835-7773feaa7945\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"
Sep 07 10:33:11.199037 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.198936    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/347a269b-3cf1-4b8c-8835-7773feaa7945-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5\" (UID: \"347a269b-3cf1-4b8c-8835-7773feaa7945\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"
Sep 07 10:33:11.199037 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.198962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcjv9\" (UniqueName: \"kubernetes.io/projected/ed22f27e-5bba-488f-8226-117195a7b0d6-kube-api-access-xcjv9\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw\" (UID: \"ed22f27e-5bba-488f-8226-117195a7b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"
Sep 07 10:33:11.199109 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.199086    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed22f27e-5bba-488f-8226-117195a7b0d6-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw\" (UID: \"ed22f27e-5bba-488f-8226-117195a7b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"
Sep 07 10:33:11.199175 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.199118    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed22f27e-5bba-488f-8226-117195a7b0d6-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw\" (UID: \"ed22f27e-5bba-488f-8226-117195a7b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"
Sep 07 10:33:11.199175 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.199162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs6rn\" (UniqueName: \"kubernetes.io/projected/347a269b-3cf1-4b8c-8835-7773feaa7945-kube-api-access-hs6rn\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5\" (UID: \"347a269b-3cf1-4b8c-8835-7773feaa7945\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"
Sep 07 10:33:11.199250 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.199225    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/347a269b-3cf1-4b8c-8835-7773feaa7945-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5\" (UID: \"347a269b-3cf1-4b8c-8835-7773feaa7945\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"
Sep 07 10:33:11.199313 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.199295    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/347a269b-3cf1-4b8c-8835-7773feaa7945-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5\" (UID: \"347a269b-3cf1-4b8c-8835-7773feaa7945\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"
Sep 07 10:33:11.208162 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.208116    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs6rn\" (UniqueName: \"kubernetes.io/projected/347a269b-3cf1-4b8c-8835-7773feaa7945-kube-api-access-hs6rn\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5\" (UID: \"347a269b-3cf1-4b8c-8835-7773feaa7945\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"
Sep 07 10:33:11.244707 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.244686    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"]
Sep 07 10:33:11.248306 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.248291    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"
Sep 07 10:33:11.258943 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.258922    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"]
Sep 07 10:33:11.299593 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.299536    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed22f27e-5bba-488f-8226-117195a7b0d6-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw\" (UID: \"ed22f27e-5bba-488f-8226-117195a7b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"
Sep 07 10:33:11.299593 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.299570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed22f27e-5bba-488f-8226-117195a7b0d6-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw\" (UID: \"ed22f27e-5bba-488f-8226-117195a7b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"
Sep 07 10:33:11.299715 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.299596    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pzcc\" (UniqueName: \"kubernetes.io/projected/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-kube-api-access-2pzcc\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm\" (UID: \"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"
Sep 07 10:33:11.299715 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.299623    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm\" (UID: \"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"
Sep 07 10:33:11.299791 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.299703    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcjv9\" (UniqueName: \"kubernetes.io/projected/ed22f27e-5bba-488f-8226-117195a7b0d6-kube-api-access-xcjv9\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw\" (UID: \"ed22f27e-5bba-488f-8226-117195a7b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"
Sep 07 10:33:11.299791 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.299746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm\" (UID: \"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"
Sep 07 10:33:11.299930 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.299917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed22f27e-5bba-488f-8226-117195a7b0d6-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw\" (UID: \"ed22f27e-5bba-488f-8226-117195a7b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"
Sep 07 10:33:11.299965 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.299953    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed22f27e-5bba-488f-8226-117195a7b0d6-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw\" (UID: \"ed22f27e-5bba-488f-8226-117195a7b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"
Sep 07 10:33:11.308105 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.308079    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcjv9\" (UniqueName: \"kubernetes.io/projected/ed22f27e-5bba-488f-8226-117195a7b0d6-kube-api-access-xcjv9\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw\" (UID: \"ed22f27e-5bba-488f-8226-117195a7b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"
Sep 07 10:33:11.346239 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.346194    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"]
Sep 07 10:33:11.349729 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.349715    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"
Sep 07 10:33:11.358197 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.358176    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"
Sep 07 10:33:11.359447 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.359421    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"]
Sep 07 10:33:11.400605 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.400575    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm\" (UID: \"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"
Sep 07 10:33:11.400733 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.400654    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pzcc\" (UniqueName: \"kubernetes.io/projected/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-kube-api-access-2pzcc\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm\" (UID: \"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"
Sep 07 10:33:11.400733 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.400701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4wnk\" (UniqueName: \"kubernetes.io/projected/023c6654-1d86-4aa8-9240-4433d6e410cc-kube-api-access-d4wnk\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7\" (UID: \"023c6654-1d86-4aa8-9240-4433d6e410cc\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"
Sep 07 10:33:11.400802 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.400734    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm\" (UID: \"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"
Sep 07 10:33:11.400802 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.400764    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/023c6654-1d86-4aa8-9240-4433d6e410cc-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7\" (UID: \"023c6654-1d86-4aa8-9240-4433d6e410cc\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"
Sep 07 10:33:11.400802 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.400789    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/023c6654-1d86-4aa8-9240-4433d6e410cc-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7\" (UID: \"023c6654-1d86-4aa8-9240-4433d6e410cc\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"
Sep 07 10:33:11.400974 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.400954    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm\" (UID: \"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"
Sep 07 10:33:11.401030 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.401013    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm\" (UID: \"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"
Sep 07 10:33:11.411047 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.411003    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pzcc\" (UniqueName: \"kubernetes.io/projected/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-kube-api-access-2pzcc\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm\" (UID: \"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"
Sep 07 10:33:11.452594 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.452569    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"
Sep 07 10:33:11.501859 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.501831    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/023c6654-1d86-4aa8-9240-4433d6e410cc-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7\" (UID: \"023c6654-1d86-4aa8-9240-4433d6e410cc\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"
Sep 07 10:33:11.501982 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.501865    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/023c6654-1d86-4aa8-9240-4433d6e410cc-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7\" (UID: \"023c6654-1d86-4aa8-9240-4433d6e410cc\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"
Sep 07 10:33:11.501982 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.501927    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4wnk\" (UniqueName: \"kubernetes.io/projected/023c6654-1d86-4aa8-9240-4433d6e410cc-kube-api-access-d4wnk\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7\" (UID: \"023c6654-1d86-4aa8-9240-4433d6e410cc\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"
Sep 07 10:33:11.502343 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.502315    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/023c6654-1d86-4aa8-9240-4433d6e410cc-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7\" (UID: \"023c6654-1d86-4aa8-9240-4433d6e410cc\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"
Sep 07 10:33:11.502441 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.502371    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/023c6654-1d86-4aa8-9240-4433d6e410cc-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7\" (UID: \"023c6654-1d86-4aa8-9240-4433d6e410cc\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"
Sep 07 10:33:11.520759 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.520737    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4wnk\" (UniqueName: \"kubernetes.io/projected/023c6654-1d86-4aa8-9240-4433d6e410cc-kube-api-access-d4wnk\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7\" (UID: \"023c6654-1d86-4aa8-9240-4433d6e410cc\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"
Sep 07 10:33:11.557694 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.557615    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"
Sep 07 10:33:11.635427 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.635399    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"]
Sep 07 10:33:11.637279 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:33:11.637245    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod347a269b_3cf1_4b8c_8835_7773feaa7945.slice/crio-f028e600a6084c51adc53855ce7e357e5bf1f31ae494e825e7ae78527dc870e3 WatchSource:0}: Error finding container f028e600a6084c51adc53855ce7e357e5bf1f31ae494e825e7ae78527dc870e3: Status 404 returned error can't find the container with id f028e600a6084c51adc53855ce7e357e5bf1f31ae494e825e7ae78527dc870e3
Sep 07 10:33:11.659301 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.659270    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"
Sep 07 10:33:11.724555 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.724525    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"]
Sep 07 10:33:11.726313 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:33:11.726283    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded22f27e_5bba_488f_8226_117195a7b0d6.slice/crio-385e406d3fb96be59ae55dd5b6026bc830d9cbfbb25fdb064a98a852cc637ece WatchSource:0}: Error finding container 385e406d3fb96be59ae55dd5b6026bc830d9cbfbb25fdb064a98a852cc637ece: Status 404 returned error can't find the container with id 385e406d3fb96be59ae55dd5b6026bc830d9cbfbb25fdb064a98a852cc637ece
Sep 07 10:33:11.835853 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.835829    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"]
Sep 07 10:33:11.837020 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:33:11.836996    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod625f91a8_06f2_4cb0_9c4b_0877c9b10ae0.slice/crio-bfa3630d52954fdc17f76e9d18eccdfa61c214c1065e68319fd5b0587ef55f2e WatchSource:0}: Error finding container bfa3630d52954fdc17f76e9d18eccdfa61c214c1065e68319fd5b0587ef55f2e: Status 404 returned error can't find the container with id bfa3630d52954fdc17f76e9d18eccdfa61c214c1065e68319fd5b0587ef55f2e
Sep 07 10:33:11.939807 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:11.939785    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"]
Sep 07 10:33:11.941517 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:33:11.941494    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod023c6654_1d86_4aa8_9240_4433d6e410cc.slice/crio-483440d53ec333ea36a9959a54315a2458ef4ff6eef80274a43496a08585faa8 WatchSource:0}: Error finding container 483440d53ec333ea36a9959a54315a2458ef4ff6eef80274a43496a08585faa8: Status 404 returned error can't find the container with id 483440d53ec333ea36a9959a54315a2458ef4ff6eef80274a43496a08585faa8
Sep 07 10:33:12.372635 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:12.372599    2577 generic.go:356] "Generic (PLEG): container finished" podID="023c6654-1d86-4aa8-9240-4433d6e410cc" containerID="d1d3beb6b246f5684da387b42f09f1d010014d974841c6f8f63d9c006043e7e6" exitCode=0
Sep 07 10:33:12.373052 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:12.372677    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7" event={"ID":"023c6654-1d86-4aa8-9240-4433d6e410cc","Type":"ContainerDied","Data":"d1d3beb6b246f5684da387b42f09f1d010014d974841c6f8f63d9c006043e7e6"}
Sep 07 10:33:12.373052 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:12.372759    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7" event={"ID":"023c6654-1d86-4aa8-9240-4433d6e410cc","Type":"ContainerStarted","Data":"483440d53ec333ea36a9959a54315a2458ef4ff6eef80274a43496a08585faa8"}
Sep 07 10:33:12.374373 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:12.374354    2577 generic.go:356] "Generic (PLEG): container finished" podID="625f91a8-06f2-4cb0-9c4b-0877c9b10ae0" containerID="6f98d117701036164bcb2ea216667fb1498577fb608da34c2038adcb2ffc518f" exitCode=0
Sep 07 10:33:12.374461 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:12.374432    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm" event={"ID":"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0","Type":"ContainerDied","Data":"6f98d117701036164bcb2ea216667fb1498577fb608da34c2038adcb2ffc518f"}
Sep 07 10:33:12.374526 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:12.374492    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm" event={"ID":"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0","Type":"ContainerStarted","Data":"bfa3630d52954fdc17f76e9d18eccdfa61c214c1065e68319fd5b0587ef55f2e"}
Sep 07 10:33:12.376041 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:12.375986    2577 generic.go:356] "Generic (PLEG): container finished" podID="347a269b-3cf1-4b8c-8835-7773feaa7945" containerID="53f6bad0be06a402f19dd47396d4dadd5c59d47fb929e771710d1f7ed6e23653" exitCode=0
Sep 07 10:33:12.376176 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:12.376063    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5" event={"ID":"347a269b-3cf1-4b8c-8835-7773feaa7945","Type":"ContainerDied","Data":"53f6bad0be06a402f19dd47396d4dadd5c59d47fb929e771710d1f7ed6e23653"}
Sep 07 10:33:12.376176 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:12.376117    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5" event={"ID":"347a269b-3cf1-4b8c-8835-7773feaa7945","Type":"ContainerStarted","Data":"f028e600a6084c51adc53855ce7e357e5bf1f31ae494e825e7ae78527dc870e3"}
Sep 07 10:33:12.378232 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:12.378198    2577 generic.go:356] "Generic (PLEG): container finished" podID="ed22f27e-5bba-488f-8226-117195a7b0d6" containerID="67156d485438f9cc7099109aa587b83efa3d66ee5a9677e369ee5a207efb1404" exitCode=0
Sep 07 10:33:12.378382 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:12.378248    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw" event={"ID":"ed22f27e-5bba-488f-8226-117195a7b0d6","Type":"ContainerDied","Data":"67156d485438f9cc7099109aa587b83efa3d66ee5a9677e369ee5a207efb1404"}
Sep 07 10:33:12.378382 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:12.378281    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw" event={"ID":"ed22f27e-5bba-488f-8226-117195a7b0d6","Type":"ContainerStarted","Data":"385e406d3fb96be59ae55dd5b6026bc830d9cbfbb25fdb064a98a852cc637ece"}
Sep 07 10:33:13.384931 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:13.384907    2577 generic.go:356] "Generic (PLEG): container finished" podID="ed22f27e-5bba-488f-8226-117195a7b0d6" containerID="fa5a2ca0f3a983e310cd968b9175046c66e8be70456a25df3894f83495a03aca" exitCode=0
Sep 07 10:33:13.385324 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:13.385003    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw" event={"ID":"ed22f27e-5bba-488f-8226-117195a7b0d6","Type":"ContainerDied","Data":"fa5a2ca0f3a983e310cd968b9175046c66e8be70456a25df3894f83495a03aca"}
Sep 07 10:33:13.387038 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:13.387017    2577 generic.go:356] "Generic (PLEG): container finished" podID="023c6654-1d86-4aa8-9240-4433d6e410cc" containerID="61c4d6b95c734ef19f53d5a19bb4037a18fc24b630f77d5a56f3a76c1c348b19" exitCode=0
Sep 07 10:33:13.387118 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:13.387051    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7" event={"ID":"023c6654-1d86-4aa8-9240-4433d6e410cc","Type":"ContainerDied","Data":"61c4d6b95c734ef19f53d5a19bb4037a18fc24b630f77d5a56f3a76c1c348b19"}
Sep 07 10:33:14.394877 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:14.394842    2577 generic.go:356] "Generic (PLEG): container finished" podID="ed22f27e-5bba-488f-8226-117195a7b0d6" containerID="8a61701acf5f60ebd7605df03a7e00e3410469297963e3f4f8064726793821fc" exitCode=0
Sep 07 10:33:14.395263 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:14.394932    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw" event={"ID":"ed22f27e-5bba-488f-8226-117195a7b0d6","Type":"ContainerDied","Data":"8a61701acf5f60ebd7605df03a7e00e3410469297963e3f4f8064726793821fc"}
Sep 07 10:33:14.397092 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:14.397060    2577 generic.go:356] "Generic (PLEG): container finished" podID="023c6654-1d86-4aa8-9240-4433d6e410cc" containerID="8106417ebb2d28b86d2ad79c75bab27b4f5382b3cf47b90a0890fb66e4fe1abf" exitCode=0
Sep 07 10:33:14.397271 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:14.397152    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7" event={"ID":"023c6654-1d86-4aa8-9240-4433d6e410cc","Type":"ContainerDied","Data":"8106417ebb2d28b86d2ad79c75bab27b4f5382b3cf47b90a0890fb66e4fe1abf"}
Sep 07 10:33:14.399036 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:14.399010    2577 generic.go:356] "Generic (PLEG): container finished" podID="347a269b-3cf1-4b8c-8835-7773feaa7945" containerID="9a1a921cf82936963d3ceb209089c956c8ded57023cad3ae91546ce043afff15" exitCode=0
Sep 07 10:33:14.399175 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:14.399033    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5" event={"ID":"347a269b-3cf1-4b8c-8835-7773feaa7945","Type":"ContainerDied","Data":"9a1a921cf82936963d3ceb209089c956c8ded57023cad3ae91546ce043afff15"}
Sep 07 10:33:15.406349 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.406316    2577 generic.go:356] "Generic (PLEG): container finished" podID="347a269b-3cf1-4b8c-8835-7773feaa7945" containerID="d52497c8ad31f08f9b1adea95233327553e17f9c53c7ab322abcfd84c9484b8e" exitCode=0
Sep 07 10:33:15.406705 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.406395    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5" event={"ID":"347a269b-3cf1-4b8c-8835-7773feaa7945","Type":"ContainerDied","Data":"d52497c8ad31f08f9b1adea95233327553e17f9c53c7ab322abcfd84c9484b8e"}
Sep 07 10:33:15.543734 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.543708    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"
Sep 07 10:33:15.564489 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.564467    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"
Sep 07 10:33:15.641370 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.641341    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcjv9\" (UniqueName: \"kubernetes.io/projected/ed22f27e-5bba-488f-8226-117195a7b0d6-kube-api-access-xcjv9\") pod \"ed22f27e-5bba-488f-8226-117195a7b0d6\" (UID: \"ed22f27e-5bba-488f-8226-117195a7b0d6\") "
Sep 07 10:33:15.641552 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.641403    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/023c6654-1d86-4aa8-9240-4433d6e410cc-bundle\") pod \"023c6654-1d86-4aa8-9240-4433d6e410cc\" (UID: \"023c6654-1d86-4aa8-9240-4433d6e410cc\") "
Sep 07 10:33:15.641552 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.641450    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4wnk\" (UniqueName: \"kubernetes.io/projected/023c6654-1d86-4aa8-9240-4433d6e410cc-kube-api-access-d4wnk\") pod \"023c6654-1d86-4aa8-9240-4433d6e410cc\" (UID: \"023c6654-1d86-4aa8-9240-4433d6e410cc\") "
Sep 07 10:33:15.641552 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.641477    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed22f27e-5bba-488f-8226-117195a7b0d6-util\") pod \"ed22f27e-5bba-488f-8226-117195a7b0d6\" (UID: \"ed22f27e-5bba-488f-8226-117195a7b0d6\") "
Sep 07 10:33:15.641552 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.641500    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed22f27e-5bba-488f-8226-117195a7b0d6-bundle\") pod \"ed22f27e-5bba-488f-8226-117195a7b0d6\" (UID: \"ed22f27e-5bba-488f-8226-117195a7b0d6\") "
Sep 07 10:33:15.641552 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.641521    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/023c6654-1d86-4aa8-9240-4433d6e410cc-util\") pod \"023c6654-1d86-4aa8-9240-4433d6e410cc\" (UID: \"023c6654-1d86-4aa8-9240-4433d6e410cc\") "
Sep 07 10:33:15.642008 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.641978    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/023c6654-1d86-4aa8-9240-4433d6e410cc-bundle" (OuterVolumeSpecName: "bundle") pod "023c6654-1d86-4aa8-9240-4433d6e410cc" (UID: "023c6654-1d86-4aa8-9240-4433d6e410cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:33:15.642008 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.641992    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed22f27e-5bba-488f-8226-117195a7b0d6-bundle" (OuterVolumeSpecName: "bundle") pod "ed22f27e-5bba-488f-8226-117195a7b0d6" (UID: "ed22f27e-5bba-488f-8226-117195a7b0d6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:33:15.644292 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.644269    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/023c6654-1d86-4aa8-9240-4433d6e410cc-kube-api-access-d4wnk" (OuterVolumeSpecName: "kube-api-access-d4wnk") pod "023c6654-1d86-4aa8-9240-4433d6e410cc" (UID: "023c6654-1d86-4aa8-9240-4433d6e410cc"). InnerVolumeSpecName "kube-api-access-d4wnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:33:15.644292 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.644279    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed22f27e-5bba-488f-8226-117195a7b0d6-kube-api-access-xcjv9" (OuterVolumeSpecName: "kube-api-access-xcjv9") pod "ed22f27e-5bba-488f-8226-117195a7b0d6" (UID: "ed22f27e-5bba-488f-8226-117195a7b0d6"). InnerVolumeSpecName "kube-api-access-xcjv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:33:15.646628 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.646590    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/023c6654-1d86-4aa8-9240-4433d6e410cc-util" (OuterVolumeSpecName: "util") pod "023c6654-1d86-4aa8-9240-4433d6e410cc" (UID: "023c6654-1d86-4aa8-9240-4433d6e410cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:33:15.647466 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.647445    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed22f27e-5bba-488f-8226-117195a7b0d6-util" (OuterVolumeSpecName: "util") pod "ed22f27e-5bba-488f-8226-117195a7b0d6" (UID: "ed22f27e-5bba-488f-8226-117195a7b0d6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:33:15.742988 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.742953    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed22f27e-5bba-488f-8226-117195a7b0d6-util\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:15.742988 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.742983    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed22f27e-5bba-488f-8226-117195a7b0d6-bundle\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:15.742988 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.742992    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/023c6654-1d86-4aa8-9240-4433d6e410cc-util\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:15.743249 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.743003    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcjv9\" (UniqueName: \"kubernetes.io/projected/ed22f27e-5bba-488f-8226-117195a7b0d6-kube-api-access-xcjv9\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:15.743249 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.743013    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/023c6654-1d86-4aa8-9240-4433d6e410cc-bundle\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:15.743249 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:15.743021    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4wnk\" (UniqueName: \"kubernetes.io/projected/023c6654-1d86-4aa8-9240-4433d6e410cc-kube-api-access-d4wnk\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:16.413125 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.413076    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw"
Sep 07 10:33:16.413125 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.413087    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a059fbw" event={"ID":"ed22f27e-5bba-488f-8226-117195a7b0d6","Type":"ContainerDied","Data":"385e406d3fb96be59ae55dd5b6026bc830d9cbfbb25fdb064a98a852cc637ece"}
Sep 07 10:33:16.413602 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.413173    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="385e406d3fb96be59ae55dd5b6026bc830d9cbfbb25fdb064a98a852cc637ece"
Sep 07 10:33:16.415177 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.415154    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7"
Sep 07 10:33:16.415288 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.415156    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv97p7" event={"ID":"023c6654-1d86-4aa8-9240-4433d6e410cc","Type":"ContainerDied","Data":"483440d53ec333ea36a9959a54315a2458ef4ff6eef80274a43496a08585faa8"}
Sep 07 10:33:16.415288 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.415272    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="483440d53ec333ea36a9959a54315a2458ef4ff6eef80274a43496a08585faa8"
Sep 07 10:33:16.544346 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.544321    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"
Sep 07 10:33:16.650393 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.650343    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hs6rn\" (UniqueName: \"kubernetes.io/projected/347a269b-3cf1-4b8c-8835-7773feaa7945-kube-api-access-hs6rn\") pod \"347a269b-3cf1-4b8c-8835-7773feaa7945\" (UID: \"347a269b-3cf1-4b8c-8835-7773feaa7945\") "
Sep 07 10:33:16.650644 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.650444    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/347a269b-3cf1-4b8c-8835-7773feaa7945-bundle\") pod \"347a269b-3cf1-4b8c-8835-7773feaa7945\" (UID: \"347a269b-3cf1-4b8c-8835-7773feaa7945\") "
Sep 07 10:33:16.650644 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.650480    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/347a269b-3cf1-4b8c-8835-7773feaa7945-util\") pod \"347a269b-3cf1-4b8c-8835-7773feaa7945\" (UID: \"347a269b-3cf1-4b8c-8835-7773feaa7945\") "
Sep 07 10:33:16.650912 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.650882    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347a269b-3cf1-4b8c-8835-7773feaa7945-bundle" (OuterVolumeSpecName: "bundle") pod "347a269b-3cf1-4b8c-8835-7773feaa7945" (UID: "347a269b-3cf1-4b8c-8835-7773feaa7945"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:33:16.652722 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.652693    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/347a269b-3cf1-4b8c-8835-7773feaa7945-kube-api-access-hs6rn" (OuterVolumeSpecName: "kube-api-access-hs6rn") pod "347a269b-3cf1-4b8c-8835-7773feaa7945" (UID: "347a269b-3cf1-4b8c-8835-7773feaa7945"). InnerVolumeSpecName "kube-api-access-hs6rn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:33:16.656090 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.656058    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347a269b-3cf1-4b8c-8835-7773feaa7945-util" (OuterVolumeSpecName: "util") pod "347a269b-3cf1-4b8c-8835-7773feaa7945" (UID: "347a269b-3cf1-4b8c-8835-7773feaa7945"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:33:16.751479 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.751450    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/347a269b-3cf1-4b8c-8835-7773feaa7945-bundle\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:16.751479 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.751475    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/347a269b-3cf1-4b8c-8835-7773feaa7945-util\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:16.751479 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:16.751484    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hs6rn\" (UniqueName: \"kubernetes.io/projected/347a269b-3cf1-4b8c-8835-7773feaa7945-kube-api-access-hs6rn\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:17.421700 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:17.421669    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5" event={"ID":"347a269b-3cf1-4b8c-8835-7773feaa7945","Type":"ContainerDied","Data":"f028e600a6084c51adc53855ce7e357e5bf1f31ae494e825e7ae78527dc870e3"}
Sep 07 10:33:17.422065 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:17.421717    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cs8zf5"
Sep 07 10:33:17.422065 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:17.421737    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f028e600a6084c51adc53855ce7e357e5bf1f31ae494e825e7ae78527dc870e3"
Sep 07 10:33:19.433351 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:19.433312    2577 generic.go:356] "Generic (PLEG): container finished" podID="625f91a8-06f2-4cb0-9c4b-0877c9b10ae0" containerID="8fb1df7b607e25a87ae43f9e2bf1abf84e4bebbcb6b123010b1e5da36c4f59d7" exitCode=0
Sep 07 10:33:19.433757 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:19.433398    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm" event={"ID":"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0","Type":"ContainerDied","Data":"8fb1df7b607e25a87ae43f9e2bf1abf84e4bebbcb6b123010b1e5da36c4f59d7"}
Sep 07 10:33:20.441866 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:20.441835    2577 generic.go:356] "Generic (PLEG): container finished" podID="625f91a8-06f2-4cb0-9c4b-0877c9b10ae0" containerID="dfcab9e3809081e91bc1abc058fcc09fc0aab878b033816b2cedf57af31a9db1" exitCode=0
Sep 07 10:33:20.442266 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:20.441898    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm" event={"ID":"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0","Type":"ContainerDied","Data":"dfcab9e3809081e91bc1abc058fcc09fc0aab878b033816b2cedf57af31a9db1"}
Sep 07 10:33:21.573572 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:21.573548    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"
Sep 07 10:33:21.693759 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:21.693724    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-bundle\") pod \"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0\" (UID: \"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0\") "
Sep 07 10:33:21.693944 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:21.693807    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pzcc\" (UniqueName: \"kubernetes.io/projected/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-kube-api-access-2pzcc\") pod \"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0\" (UID: \"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0\") "
Sep 07 10:33:21.693944 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:21.693828    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-util\") pod \"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0\" (UID: \"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0\") "
Sep 07 10:33:21.694594 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:21.694560    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-bundle" (OuterVolumeSpecName: "bundle") pod "625f91a8-06f2-4cb0-9c4b-0877c9b10ae0" (UID: "625f91a8-06f2-4cb0-9c4b-0877c9b10ae0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:33:21.696006 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:21.695983    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-kube-api-access-2pzcc" (OuterVolumeSpecName: "kube-api-access-2pzcc") pod "625f91a8-06f2-4cb0-9c4b-0877c9b10ae0" (UID: "625f91a8-06f2-4cb0-9c4b-0877c9b10ae0"). InnerVolumeSpecName "kube-api-access-2pzcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:33:21.700316 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:21.700259    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-util" (OuterVolumeSpecName: "util") pod "625f91a8-06f2-4cb0-9c4b-0877c9b10ae0" (UID: "625f91a8-06f2-4cb0-9c4b-0877c9b10ae0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:33:21.794320 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:21.794286    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-bundle\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:21.794464 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:21.794316    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pzcc\" (UniqueName: \"kubernetes.io/projected/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-kube-api-access-2pzcc\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:21.794464 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:21.794350    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/625f91a8-06f2-4cb0-9c4b-0877c9b10ae0-util\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:22.454486 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:22.454449    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm" event={"ID":"625f91a8-06f2-4cb0-9c4b-0877c9b10ae0","Type":"ContainerDied","Data":"bfa3630d52954fdc17f76e9d18eccdfa61c214c1065e68319fd5b0587ef55f2e"}
Sep 07 10:33:22.454665 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:22.454497    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bfa3630d52954fdc17f76e9d18eccdfa61c214c1065e68319fd5b0587ef55f2e"
Sep 07 10:33:22.454665 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:22.454462    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053v6vsm"
Sep 07 10:33:25.475013 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:25.474981    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:26.275982 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:26.275939    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:28.770110 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770075    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-cffps"]
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770400    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="347a269b-3cf1-4b8c-8835-7773feaa7945" containerName="pull"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770411    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="347a269b-3cf1-4b8c-8835-7773feaa7945" containerName="pull"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770422    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="023c6654-1d86-4aa8-9240-4433d6e410cc" containerName="util"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770427    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="023c6654-1d86-4aa8-9240-4433d6e410cc" containerName="util"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770432    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed22f27e-5bba-488f-8226-117195a7b0d6" containerName="pull"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770437    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed22f27e-5bba-488f-8226-117195a7b0d6" containerName="pull"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770444    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="347a269b-3cf1-4b8c-8835-7773feaa7945" containerName="extract"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770450    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="347a269b-3cf1-4b8c-8835-7773feaa7945" containerName="extract"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770456    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed22f27e-5bba-488f-8226-117195a7b0d6" containerName="util"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770461    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed22f27e-5bba-488f-8226-117195a7b0d6" containerName="util"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770471    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="347a269b-3cf1-4b8c-8835-7773feaa7945" containerName="util"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770475    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="347a269b-3cf1-4b8c-8835-7773feaa7945" containerName="util"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770480    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="625f91a8-06f2-4cb0-9c4b-0877c9b10ae0" containerName="pull"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770484    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="625f91a8-06f2-4cb0-9c4b-0877c9b10ae0" containerName="pull"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770490    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="625f91a8-06f2-4cb0-9c4b-0877c9b10ae0" containerName="util"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770494    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="625f91a8-06f2-4cb0-9c4b-0877c9b10ae0" containerName="util"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770502    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="023c6654-1d86-4aa8-9240-4433d6e410cc" containerName="pull"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770506    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="023c6654-1d86-4aa8-9240-4433d6e410cc" containerName="pull"
Sep 07 10:33:28.770506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770514    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed22f27e-5bba-488f-8226-117195a7b0d6" containerName="extract"
Sep 07 10:33:28.771052 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770519    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed22f27e-5bba-488f-8226-117195a7b0d6" containerName="extract"
Sep 07 10:33:28.771052 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770526    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="023c6654-1d86-4aa8-9240-4433d6e410cc" containerName="extract"
Sep 07 10:33:28.771052 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770531    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="023c6654-1d86-4aa8-9240-4433d6e410cc" containerName="extract"
Sep 07 10:33:28.771052 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770536    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="625f91a8-06f2-4cb0-9c4b-0877c9b10ae0" containerName="extract"
Sep 07 10:33:28.771052 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770541    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="625f91a8-06f2-4cb0-9c4b-0877c9b10ae0" containerName="extract"
Sep 07 10:33:28.771052 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770590    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="347a269b-3cf1-4b8c-8835-7773feaa7945" containerName="extract"
Sep 07 10:33:28.771052 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770598    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="023c6654-1d86-4aa8-9240-4433d6e410cc" containerName="extract"
Sep 07 10:33:28.771052 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770603    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed22f27e-5bba-488f-8226-117195a7b0d6" containerName="extract"
Sep 07 10:33:28.771052 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.770609    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="625f91a8-06f2-4cb0-9c4b-0877c9b10ae0" containerName="extract"
Sep 07 10:33:28.773766 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.773750    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-cffps"
Sep 07 10:33:28.776939 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.776920    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5zl8m\""
Sep 07 10:33:28.777276 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.777256    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 07 10:33:28.777836 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.777821    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 07 10:33:28.792506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.792483    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-cffps"]
Sep 07 10:33:28.855796 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.855762    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pmpj\" (UniqueName: \"kubernetes.io/projected/6276af5a-6445-4829-9fdf-ab1262b939bf-kube-api-access-7pmpj\") pod \"authorino-operator-68d88948d8-cffps\" (UID: \"6276af5a-6445-4829-9fdf-ab1262b939bf\") " pod="kuadrant-system/authorino-operator-68d88948d8-cffps"
Sep 07 10:33:28.956784 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.956735    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pmpj\" (UniqueName: \"kubernetes.io/projected/6276af5a-6445-4829-9fdf-ab1262b939bf-kube-api-access-7pmpj\") pod \"authorino-operator-68d88948d8-cffps\" (UID: \"6276af5a-6445-4829-9fdf-ab1262b939bf\") " pod="kuadrant-system/authorino-operator-68d88948d8-cffps"
Sep 07 10:33:28.968303 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:28.968271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pmpj\" (UniqueName: \"kubernetes.io/projected/6276af5a-6445-4829-9fdf-ab1262b939bf-kube-api-access-7pmpj\") pod \"authorino-operator-68d88948d8-cffps\" (UID: \"6276af5a-6445-4829-9fdf-ab1262b939bf\") " pod="kuadrant-system/authorino-operator-68d88948d8-cffps"
Sep 07 10:33:29.086088 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:29.086002    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-cffps"
Sep 07 10:33:29.369456 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:29.369430    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-cffps"]
Sep 07 10:33:29.371371 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:33:29.371341    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6276af5a_6445_4829_9fdf_ab1262b939bf.slice/crio-e35a433cc66b7c8957379223358d90b0f43cabbe724f409105ca98d246296b26 WatchSource:0}: Error finding container e35a433cc66b7c8957379223358d90b0f43cabbe724f409105ca98d246296b26: Status 404 returned error can't find the container with id e35a433cc66b7c8957379223358d90b0f43cabbe724f409105ca98d246296b26
Sep 07 10:33:29.489761 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:29.489732    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-cffps" event={"ID":"6276af5a-6445-4829-9fdf-ab1262b939bf","Type":"ContainerStarted","Data":"e35a433cc66b7c8957379223358d90b0f43cabbe724f409105ca98d246296b26"}
Sep 07 10:33:29.539777 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:29.539748    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:29.724929 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:29.724888    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:30.984624 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:30.984588    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-dk5n6"]
Sep 07 10:33:30.993056 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:30.993031    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-dk5n6"
Sep 07 10:33:30.995764 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:30.995714    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 07 10:33:30.995899 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:30.995863    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5qfg8\""
Sep 07 10:33:31.002554 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:31.002332    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-dk5n6"]
Sep 07 10:33:31.075270 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:31.075230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvmph\" (UniqueName: \"kubernetes.io/projected/9f7a4c3b-1dad-4023-8695-0539a0840895-kube-api-access-vvmph\") pod \"dns-operator-controller-manager-5985564776-dk5n6\" (UID: \"9f7a4c3b-1dad-4023-8695-0539a0840895\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-dk5n6"
Sep 07 10:33:31.174662 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:31.174626    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:31.175698 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:31.175673    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvmph\" (UniqueName: \"kubernetes.io/projected/9f7a4c3b-1dad-4023-8695-0539a0840895-kube-api-access-vvmph\") pod \"dns-operator-controller-manager-5985564776-dk5n6\" (UID: \"9f7a4c3b-1dad-4023-8695-0539a0840895\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-dk5n6"
Sep 07 10:33:31.212858 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:31.212835    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvmph\" (UniqueName: \"kubernetes.io/projected/9f7a4c3b-1dad-4023-8695-0539a0840895-kube-api-access-vvmph\") pod \"dns-operator-controller-manager-5985564776-dk5n6\" (UID: \"9f7a4c3b-1dad-4023-8695-0539a0840895\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-dk5n6"
Sep 07 10:33:31.307513 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:31.307429    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-dk5n6"
Sep 07 10:33:31.691595 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:31.691563    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:31.942239 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:31.942215    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-dk5n6"]
Sep 07 10:33:31.944037 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:33:31.944004    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f7a4c3b_1dad_4023_8695_0539a0840895.slice/crio-dce5cd97217ab0dd9507248563405b26657b35863bd02e5ecf9851c4bbb7aa6a WatchSource:0}: Error finding container dce5cd97217ab0dd9507248563405b26657b35863bd02e5ecf9851c4bbb7aa6a: Status 404 returned error can't find the container with id dce5cd97217ab0dd9507248563405b26657b35863bd02e5ecf9851c4bbb7aa6a
Sep 07 10:33:32.105464 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:32.105438    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:32.486230 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:32.486192    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:32.508447 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:32.508421    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-cffps" event={"ID":"6276af5a-6445-4829-9fdf-ab1262b939bf","Type":"ContainerStarted","Data":"775773ce667edb905b13a475a60f2a442fb36a6a7cf43c48415a212937aa6742"}
Sep 07 10:33:32.508603 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:32.508475    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-cffps"
Sep 07 10:33:32.509813 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:32.509788    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-dk5n6" event={"ID":"9f7a4c3b-1dad-4023-8695-0539a0840895","Type":"ContainerStarted","Data":"dce5cd97217ab0dd9507248563405b26657b35863bd02e5ecf9851c4bbb7aa6a"}
Sep 07 10:33:32.528041 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:32.528003    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-cffps" podStartSLOduration=2.160967855 podStartE2EDuration="4.527989471s" totalImagesPullingTime="2.367021616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:33:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:33:32.526402391 +0000 UTC m=+816.260576615" watchObservedRunningTime="2026-09-07 10:33:32.527989471 +0000 UTC m=+816.262163708"
Sep 07 10:33:32.955449 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:32.955414    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:33.080380 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:33.080345    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:34.526202 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:34.526169    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-dk5n6" event={"ID":"9f7a4c3b-1dad-4023-8695-0539a0840895","Type":"ContainerStarted","Data":"088065d0777a52c59cfe4f3f4106f1ca5b4db249ef28e2fb1adf48886b3706a8"}
Sep 07 10:33:34.526477 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:34.526262    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-dk5n6"
Sep 07 10:33:34.548079 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:34.548032    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-dk5n6" podStartSLOduration=2.046631385 podStartE2EDuration="4.548020178s" totalImagesPullingTime="2.501388793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:33:34.544013605 +0000 UTC m=+818.278187851" watchObservedRunningTime="2026-09-07 10:33:34.548020178 +0000 UTC m=+818.282194412"
Sep 07 10:33:35.397584 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:35.397550    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"]
Sep 07 10:33:35.401105 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:35.401083    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:35.403914 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:35.403890    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6h7k6\""
Sep 07 10:33:35.418068 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:35.418034    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"]
Sep 07 10:33:35.515308 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:35.515273    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cwll\" (UniqueName: \"kubernetes.io/projected/a3bad773-1e4c-41a6-a1f6-afd3074b3d18-kube-api-access-4cwll\") pod \"kuadrant-operator-controller-manager-58fc56757c-nfpvb\" (UID: \"a3bad773-1e4c-41a6-a1f6-afd3074b3d18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:35.515495 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:35.515323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3bad773-1e4c-41a6-a1f6-afd3074b3d18-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-nfpvb\" (UID: \"a3bad773-1e4c-41a6-a1f6-afd3074b3d18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:35.616564 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:35.616525    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cwll\" (UniqueName: \"kubernetes.io/projected/a3bad773-1e4c-41a6-a1f6-afd3074b3d18-kube-api-access-4cwll\") pod \"kuadrant-operator-controller-manager-58fc56757c-nfpvb\" (UID: \"a3bad773-1e4c-41a6-a1f6-afd3074b3d18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:35.616983 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:35.616577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3bad773-1e4c-41a6-a1f6-afd3074b3d18-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-nfpvb\" (UID: \"a3bad773-1e4c-41a6-a1f6-afd3074b3d18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:35.617032 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:35.617012    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3bad773-1e4c-41a6-a1f6-afd3074b3d18-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-nfpvb\" (UID: \"a3bad773-1e4c-41a6-a1f6-afd3074b3d18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:35.627450 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:35.627424    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cwll\" (UniqueName: \"kubernetes.io/projected/a3bad773-1e4c-41a6-a1f6-afd3074b3d18-kube-api-access-4cwll\") pod \"kuadrant-operator-controller-manager-58fc56757c-nfpvb\" (UID: \"a3bad773-1e4c-41a6-a1f6-afd3074b3d18\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:35.712047 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:35.712015    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:35.899275 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:35.899234    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:35.997214 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:35.997191    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"]
Sep 07 10:33:35.998895 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:33:35.998861    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3bad773_1e4c_41a6_a1f6_afd3074b3d18.slice/crio-bafc1217114343b8ed3d84e4f5a8ab4ebd846f22e6db09a44e2963f45ee85ab5 WatchSource:0}: Error finding container bafc1217114343b8ed3d84e4f5a8ab4ebd846f22e6db09a44e2963f45ee85ab5: Status 404 returned error can't find the container with id bafc1217114343b8ed3d84e4f5a8ab4ebd846f22e6db09a44e2963f45ee85ab5
Sep 07 10:33:36.537934 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:36.537899    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb" event={"ID":"a3bad773-1e4c-41a6-a1f6-afd3074b3d18","Type":"ContainerStarted","Data":"bafc1217114343b8ed3d84e4f5a8ab4ebd846f22e6db09a44e2963f45ee85ab5"}
Sep 07 10:33:37.337001 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:37.336967    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:37.364699 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:37.364661    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"]
Sep 07 10:33:37.368654 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:37.368623    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:37.371856 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:37.371835    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-xv9nq\""
Sep 07 10:33:37.392883 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:37.392849    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"]
Sep 07 10:33:37.432238 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:37.432203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqqq8\" (UniqueName: \"kubernetes.io/projected/1d8fefac-f6f9-4794-b1cd-31984f60fa83-kube-api-access-zqqq8\") pod \"limitador-operator-controller-manager-6c549b6d85-zwkxl\" (UID: \"1d8fefac-f6f9-4794-b1cd-31984f60fa83\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:37.533516 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:37.533481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqqq8\" (UniqueName: \"kubernetes.io/projected/1d8fefac-f6f9-4794-b1cd-31984f60fa83-kube-api-access-zqqq8\") pod \"limitador-operator-controller-manager-6c549b6d85-zwkxl\" (UID: \"1d8fefac-f6f9-4794-b1cd-31984f60fa83\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:37.543949 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:37.543913    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqqq8\" (UniqueName: \"kubernetes.io/projected/1d8fefac-f6f9-4794-b1cd-31984f60fa83-kube-api-access-zqqq8\") pod \"limitador-operator-controller-manager-6c549b6d85-zwkxl\" (UID: \"1d8fefac-f6f9-4794-b1cd-31984f60fa83\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:37.682249 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:37.682152    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:37.973458 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:37.973427    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"]
Sep 07 10:33:37.976021 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:33:37.975991    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d8fefac_f6f9_4794_b1cd_31984f60fa83.slice/crio-dd3cc0d996e1089a5e58d818b049ab7977e17b1fdd3842fd28fb4765e29744ad WatchSource:0}: Error finding container dd3cc0d996e1089a5e58d818b049ab7977e17b1fdd3842fd28fb4765e29744ad: Status 404 returned error can't find the container with id dd3cc0d996e1089a5e58d818b049ab7977e17b1fdd3842fd28fb4765e29744ad
Sep 07 10:33:38.557315 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:38.557272    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl" event={"ID":"1d8fefac-f6f9-4794-b1cd-31984f60fa83","Type":"ContainerStarted","Data":"dd3cc0d996e1089a5e58d818b049ab7977e17b1fdd3842fd28fb4765e29744ad"}
Sep 07 10:33:38.887924 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:38.887837    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:38.993713 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:38.993680    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:39.299967 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:39.299929    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:39.517123 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:39.517075    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:40.574255 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:40.574219    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb" event={"ID":"a3bad773-1e4c-41a6-a1f6-afd3074b3d18","Type":"ContainerStarted","Data":"f277e3be4cb453bfc9a11affa310c0053689bff716e7ccef75349a8cd0c75ccb"}
Sep 07 10:33:40.574736 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:40.574373    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:40.599452 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:40.599400    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb" podStartSLOduration=1.628488709 podStartE2EDuration="5.599383527s" totalImagesPullingTime="3.970894818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:33:40.596116504 +0000 UTC m=+824.330290739" watchObservedRunningTime="2026-09-07 10:33:40.599383527 +0000 UTC m=+824.333557760"
Sep 07 10:33:41.582032 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:41.581995    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl" event={"ID":"1d8fefac-f6f9-4794-b1cd-31984f60fa83","Type":"ContainerStarted","Data":"204a1825583a57ea478c5447dfe3cabe8311a4c8e9bdda4632bba2047ee4dd03"}
Sep 07 10:33:41.582486 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:41.582177    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:41.601998 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:41.601946    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl" podStartSLOduration=1.8656856309999998 podStartE2EDuration="4.601928703s" totalImagesPullingTime="2.736243072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:33:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:33:41.601013774 +0000 UTC m=+825.335188020" watchObservedRunningTime="2026-09-07 10:33:41.601928703 +0000 UTC m=+825.336102935"
Sep 07 10:33:42.662799 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.662762    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh"]
Sep 07 10:33:42.665860 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.665839    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh"
Sep 07 10:33:42.668194 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.668161    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:42.668444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.668426    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vzl6n\""
Sep 07 10:33:42.668543 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.668469    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 07 10:33:42.668543 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.668483    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 07 10:33:42.676898 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.676876    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh"]
Sep 07 10:33:42.762025 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.761995    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:42.777635 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.777608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/311bf9fc-4c14-4211-a929-4e0142ca0df4-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-hzrmh\" (UID: \"311bf9fc-4c14-4211-a929-4e0142ca0df4\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh"
Sep 07 10:33:42.777805 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.777648    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/311bf9fc-4c14-4211-a929-4e0142ca0df4-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-hzrmh\" (UID: \"311bf9fc-4c14-4211-a929-4e0142ca0df4\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh"
Sep 07 10:33:42.777805 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.777668    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4kzg\" (UniqueName: \"kubernetes.io/projected/311bf9fc-4c14-4211-a929-4e0142ca0df4-kube-api-access-l4kzg\") pod \"kuadrant-console-plugin-598f86468b-hzrmh\" (UID: \"311bf9fc-4c14-4211-a929-4e0142ca0df4\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh"
Sep 07 10:33:42.878700 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.878661    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/311bf9fc-4c14-4211-a929-4e0142ca0df4-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-hzrmh\" (UID: \"311bf9fc-4c14-4211-a929-4e0142ca0df4\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh"
Sep 07 10:33:42.878911 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.878711    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/311bf9fc-4c14-4211-a929-4e0142ca0df4-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-hzrmh\" (UID: \"311bf9fc-4c14-4211-a929-4e0142ca0df4\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh"
Sep 07 10:33:42.878911 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.878729    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4kzg\" (UniqueName: \"kubernetes.io/projected/311bf9fc-4c14-4211-a929-4e0142ca0df4-kube-api-access-l4kzg\") pod \"kuadrant-console-plugin-598f86468b-hzrmh\" (UID: \"311bf9fc-4c14-4211-a929-4e0142ca0df4\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh"
Sep 07 10:33:42.879449 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.879425    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/311bf9fc-4c14-4211-a929-4e0142ca0df4-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-hzrmh\" (UID: \"311bf9fc-4c14-4211-a929-4e0142ca0df4\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh"
Sep 07 10:33:42.881384 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.881359    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/311bf9fc-4c14-4211-a929-4e0142ca0df4-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-hzrmh\" (UID: \"311bf9fc-4c14-4211-a929-4e0142ca0df4\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh"
Sep 07 10:33:42.886619 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.886592    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:42.889244 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.889224    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4kzg\" (UniqueName: \"kubernetes.io/projected/311bf9fc-4c14-4211-a929-4e0142ca0df4-kube-api-access-l4kzg\") pod \"kuadrant-console-plugin-598f86468b-hzrmh\" (UID: \"311bf9fc-4c14-4211-a929-4e0142ca0df4\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh"
Sep 07 10:33:42.975863 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.975827    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh"
Sep 07 10:33:42.987320 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:42.987276    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:43.252952 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:43.252925    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh"]
Sep 07 10:33:43.254723 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:33:43.254698    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod311bf9fc_4c14_4211_a929_4e0142ca0df4.slice/crio-d7a8c8903c25d0ead09e13d30f538c56b1f9fba282105489ed3c9ddbbb857a3a WatchSource:0}: Error finding container d7a8c8903c25d0ead09e13d30f538c56b1f9fba282105489ed3c9ddbbb857a3a: Status 404 returned error can't find the container with id d7a8c8903c25d0ead09e13d30f538c56b1f9fba282105489ed3c9ddbbb857a3a
Sep 07 10:33:43.522393 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:43.522293    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-cffps"
Sep 07 10:33:43.594866 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:43.594827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh" event={"ID":"311bf9fc-4c14-4211-a929-4e0142ca0df4","Type":"ContainerStarted","Data":"d7a8c8903c25d0ead09e13d30f538c56b1f9fba282105489ed3c9ddbbb857a3a"}
Sep 07 10:33:45.535266 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:45.535231    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-dk5n6"
Sep 07 10:33:48.499734 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:48.499701    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:48.603155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:48.603107    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:48.625618 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:48.625591    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh" event={"ID":"311bf9fc-4c14-4211-a929-4e0142ca0df4","Type":"ContainerStarted","Data":"2d224d62d00b2c247521e25d6b8f2903fd663a8ba3e7a2cc173de35c56732227"}
Sep 07 10:33:48.645220 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:48.645174    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-hzrmh" podStartSLOduration=2.304013446 podStartE2EDuration="6.645154301s" totalImagesPullingTime="4.341140855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:33:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:33:48.643603346 +0000 UTC m=+832.377777578" watchObservedRunningTime="2026-09-07 10:33:48.645154301 +0000 UTC m=+832.379328532"
Sep 07 10:33:48.706307 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:48.706278    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:51.584909 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:51.584875    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:52.590285 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:52.590254    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:53.335241 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.335204    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"]
Sep 07 10:33:53.338705 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.338689    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"
Sep 07 10:33:53.351946 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.351919    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"]
Sep 07 10:33:53.376561 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.376533    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/02b854d5-537d-4d3c-ab23-ec1a5c8b942b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-c6fvc\" (UID: \"02b854d5-537d-4d3c-ab23-ec1a5c8b942b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"
Sep 07 10:33:53.376687 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.376574    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z4ss\" (UniqueName: \"kubernetes.io/projected/02b854d5-537d-4d3c-ab23-ec1a5c8b942b-kube-api-access-8z4ss\") pod \"kuadrant-operator-controller-manager-68cf589589-c6fvc\" (UID: \"02b854d5-537d-4d3c-ab23-ec1a5c8b942b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"
Sep 07 10:33:53.477967 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.477934    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/02b854d5-537d-4d3c-ab23-ec1a5c8b942b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-c6fvc\" (UID: \"02b854d5-537d-4d3c-ab23-ec1a5c8b942b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"
Sep 07 10:33:53.478119 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.477978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z4ss\" (UniqueName: \"kubernetes.io/projected/02b854d5-537d-4d3c-ab23-ec1a5c8b942b-kube-api-access-8z4ss\") pod \"kuadrant-operator-controller-manager-68cf589589-c6fvc\" (UID: \"02b854d5-537d-4d3c-ab23-ec1a5c8b942b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"
Sep 07 10:33:53.478372 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.478348    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/02b854d5-537d-4d3c-ab23-ec1a5c8b942b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-c6fvc\" (UID: \"02b854d5-537d-4d3c-ab23-ec1a5c8b942b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"
Sep 07 10:33:53.487370 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.487351    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z4ss\" (UniqueName: \"kubernetes.io/projected/02b854d5-537d-4d3c-ab23-ec1a5c8b942b-kube-api-access-8z4ss\") pod \"kuadrant-operator-controller-manager-68cf589589-c6fvc\" (UID: \"02b854d5-537d-4d3c-ab23-ec1a5c8b942b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"
Sep 07 10:33:53.649073 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.648983    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"
Sep 07 10:33:53.780213 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.780174    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"]
Sep 07 10:33:53.780394 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.780371    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb" podUID="a3bad773-1e4c-41a6-a1f6-afd3074b3d18" containerName="manager" containerID="cri-o://f277e3be4cb453bfc9a11affa310c0053689bff716e7ccef75349a8cd0c75ccb" gracePeriod=2
Sep 07 10:33:53.790112 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.790086    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"]
Sep 07 10:33:53.802835 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.802810    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"]
Sep 07 10:33:53.808896 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.808873    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"]
Sep 07 10:33:53.809487 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.809467    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3bad773-1e4c-41a6-a1f6-afd3074b3d18" containerName="manager"
Sep 07 10:33:53.809559 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.809491    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3bad773-1e4c-41a6-a1f6-afd3074b3d18" containerName="manager"
Sep 07 10:33:53.809601 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.809585    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3bad773-1e4c-41a6-a1f6-afd3074b3d18" containerName="manager"
Sep 07 10:33:53.813174 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.813155    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"
Sep 07 10:33:53.814344 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.814320    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"]
Sep 07 10:33:53.815376 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:53.815343    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-nfpvb\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="a3bad773-1e4c-41a6-a1f6-afd3074b3d18" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:53.828430 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.828397    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"]
Sep 07 10:33:53.830800 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.830778    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"]
Sep 07 10:33:53.830976 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.830956    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl" podUID="1d8fefac-f6f9-4794-b1cd-31984f60fa83" containerName="manager" containerID="cri-o://204a1825583a57ea478c5447dfe3cabe8311a4c8e9bdda4632bba2047ee4dd03" gracePeriod=2
Sep 07 10:33:53.836261 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.836236    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"]
Sep 07 10:33:53.839871 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.839760    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"
Sep 07 10:33:53.842424 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.842401    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"]
Sep 07 10:33:53.866465 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:53.866431    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-zwkxl\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="1d8fefac-f6f9-4794-b1cd-31984f60fa83" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:53.870102 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:53.870075    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-nfpvb\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="a3bad773-1e4c-41a6-a1f6-afd3074b3d18" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:53.873084 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.873054    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"]
Sep 07 10:33:53.881171 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.881117    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fszws\" (UniqueName: \"kubernetes.io/projected/1a0ef1a5-fbda-423e-b224-3c28c1121a7d-kube-api-access-fszws\") pod \"kuadrant-operator-controller-manager-68cf589589-qzc7f\" (UID: \"1a0ef1a5-fbda-423e-b224-3c28c1121a7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"
Sep 07 10:33:53.881272 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.881239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a0ef1a5-fbda-423e-b224-3c28c1121a7d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-qzc7f\" (UID: \"1a0ef1a5-fbda-423e-b224-3c28c1121a7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"
Sep 07 10:33:53.889989 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.889967    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtn57"]
Sep 07 10:33:53.890503 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.890483    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d8fefac-f6f9-4794-b1cd-31984f60fa83" containerName="manager"
Sep 07 10:33:53.890693 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.890507    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d8fefac-f6f9-4794-b1cd-31984f60fa83" containerName="manager"
Sep 07 10:33:53.890693 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.890678    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d8fefac-f6f9-4794-b1cd-31984f60fa83" containerName="manager"
Sep 07 10:33:53.898792 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.898761    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtn57"
Sep 07 10:33:53.909991 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.909834    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtn57"]
Sep 07 10:33:53.926894 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:53.926865    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-zwkxl\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="1d8fefac-f6f9-4794-b1cd-31984f60fa83" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:53.929759 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:53.929725    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-nfpvb\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="a3bad773-1e4c-41a6-a1f6-afd3074b3d18" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:53.983215 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.982840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hpch\" (UniqueName: \"kubernetes.io/projected/1d4c650e-31e8-427a-9178-4afa96b2a359-kube-api-access-6hpch\") pod \"kuadrant-operator-controller-manager-58fc56757c-5wlsh\" (UID: \"1d4c650e-31e8-427a-9178-4afa96b2a359\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"
Sep 07 10:33:53.983215 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.982875    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvnrm\" (UniqueName: \"kubernetes.io/projected/c0c4bdd8-1490-426c-b5c9-aad3aa2deb6a-kube-api-access-mvnrm\") pod \"limitador-operator-controller-manager-6c549b6d85-vtn57\" (UID: \"c0c4bdd8-1490-426c-b5c9-aad3aa2deb6a\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtn57"
Sep 07 10:33:53.983215 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.982906    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d4c650e-31e8-427a-9178-4afa96b2a359-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-5wlsh\" (UID: \"1d4c650e-31e8-427a-9178-4afa96b2a359\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"
Sep 07 10:33:53.983215 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.982941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fszws\" (UniqueName: \"kubernetes.io/projected/1a0ef1a5-fbda-423e-b224-3c28c1121a7d-kube-api-access-fszws\") pod \"kuadrant-operator-controller-manager-68cf589589-qzc7f\" (UID: \"1a0ef1a5-fbda-423e-b224-3c28c1121a7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"
Sep 07 10:33:53.983215 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.982972    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a0ef1a5-fbda-423e-b224-3c28c1121a7d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-qzc7f\" (UID: \"1a0ef1a5-fbda-423e-b224-3c28c1121a7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"
Sep 07 10:33:53.983609 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.983411    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a0ef1a5-fbda-423e-b224-3c28c1121a7d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-qzc7f\" (UID: \"1a0ef1a5-fbda-423e-b224-3c28c1121a7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"
Sep 07 10:33:53.994275 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:53.994243    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fszws\" (UniqueName: \"kubernetes.io/projected/1a0ef1a5-fbda-423e-b224-3c28c1121a7d-kube-api-access-fszws\") pod \"kuadrant-operator-controller-manager-68cf589589-qzc7f\" (UID: \"1a0ef1a5-fbda-423e-b224-3c28c1121a7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"
Sep 07 10:33:54.058419 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:54.058385    2577 log.go:32] "RunPodSandbox from runtime service failed" err=<
Sep 07 10:33:54.058419 ip-10-0-128-166 kubenswrapper[2577]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-68cf589589-c6fvc_kuadrant-system_02b854d5-537d-4d3c-ab23-ec1a5c8b942b_0(d5a4fe77165f92f5621959f70125b7f9842d87af5aa1d1753d73d805fcbe1d88): error adding pod kuadrant-system_kuadrant-operator-controller-manager-68cf589589-c6fvc to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"d5a4fe77165f92f5621959f70125b7f9842d87af5aa1d1753d73d805fcbe1d88" Netns:"/var/run/netns/cfcb1864-3797-4625-bddd-467c40afc229" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-68cf589589-c6fvc;K8S_POD_INFRA_CONTAINER_ID=d5a4fe77165f92f5621959f70125b7f9842d87af5aa1d1753d73d805fcbe1d88;K8S_POD_UID=02b854d5-537d-4d3c-ab23-ec1a5c8b942b" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc] networking: [kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc/02b854d5-537d-4d3c-ab23-ec1a5c8b942b:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc d5a4fe77165f92f5621959f70125b7f9842d87af5aa1d1753d73d805fcbe1d88 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:84:00:2f [10.132.0.47/23]
Sep 07 10:33:54.058419 ip-10-0-128-166 kubenswrapper[2577]:         '
Sep 07 10:33:54.058419 ip-10-0-128-166 kubenswrapper[2577]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 07 10:33:54.058419 ip-10-0-128-166 kubenswrapper[2577]:  >
Sep 07 10:33:54.058631 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:54.058447    2577 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=<
Sep 07 10:33:54.058631 ip-10-0-128-166 kubenswrapper[2577]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-68cf589589-c6fvc_kuadrant-system_02b854d5-537d-4d3c-ab23-ec1a5c8b942b_0(d5a4fe77165f92f5621959f70125b7f9842d87af5aa1d1753d73d805fcbe1d88): error adding pod kuadrant-system_kuadrant-operator-controller-manager-68cf589589-c6fvc to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"d5a4fe77165f92f5621959f70125b7f9842d87af5aa1d1753d73d805fcbe1d88" Netns:"/var/run/netns/cfcb1864-3797-4625-bddd-467c40afc229" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-68cf589589-c6fvc;K8S_POD_INFRA_CONTAINER_ID=d5a4fe77165f92f5621959f70125b7f9842d87af5aa1d1753d73d805fcbe1d88;K8S_POD_UID=02b854d5-537d-4d3c-ab23-ec1a5c8b942b" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc] networking: [kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc/02b854d5-537d-4d3c-ab23-ec1a5c8b942b:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc d5a4fe77165f92f5621959f70125b7f9842d87af5aa1d1753d73d805fcbe1d88 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:84:00:2f [10.132.0.47/23]
Sep 07 10:33:54.058631 ip-10-0-128-166 kubenswrapper[2577]:         '
Sep 07 10:33:54.058631 ip-10-0-128-166 kubenswrapper[2577]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 07 10:33:54.058631 ip-10-0-128-166 kubenswrapper[2577]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"
Sep 07 10:33:54.068815 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.068795    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:54.071482 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:54.071459    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-nfpvb\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="a3bad773-1e4c-41a6-a1f6-afd3074b3d18" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:54.071660 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.071645    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:54.073459 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:54.073438    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-zwkxl\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="1d8fefac-f6f9-4794-b1cd-31984f60fa83" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:54.075342 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:54.075324    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-zwkxl\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="1d8fefac-f6f9-4794-b1cd-31984f60fa83" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:54.077690 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:54.077671    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-nfpvb\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="a3bad773-1e4c-41a6-a1f6-afd3074b3d18" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:54.083873 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.083850    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3bad773-1e4c-41a6-a1f6-afd3074b3d18-extensions-socket-volume\") pod \"a3bad773-1e4c-41a6-a1f6-afd3074b3d18\" (UID: \"a3bad773-1e4c-41a6-a1f6-afd3074b3d18\") "
Sep 07 10:33:54.083952 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.083891    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqqq8\" (UniqueName: \"kubernetes.io/projected/1d8fefac-f6f9-4794-b1cd-31984f60fa83-kube-api-access-zqqq8\") pod \"1d8fefac-f6f9-4794-b1cd-31984f60fa83\" (UID: \"1d8fefac-f6f9-4794-b1cd-31984f60fa83\") "
Sep 07 10:33:54.083952 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.083939    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cwll\" (UniqueName: \"kubernetes.io/projected/a3bad773-1e4c-41a6-a1f6-afd3074b3d18-kube-api-access-4cwll\") pod \"a3bad773-1e4c-41a6-a1f6-afd3074b3d18\" (UID: \"a3bad773-1e4c-41a6-a1f6-afd3074b3d18\") "
Sep 07 10:33:54.084172 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.084151    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hpch\" (UniqueName: \"kubernetes.io/projected/1d4c650e-31e8-427a-9178-4afa96b2a359-kube-api-access-6hpch\") pod \"kuadrant-operator-controller-manager-58fc56757c-5wlsh\" (UID: \"1d4c650e-31e8-427a-9178-4afa96b2a359\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"
Sep 07 10:33:54.084270 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.084188    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvnrm\" (UniqueName: \"kubernetes.io/projected/c0c4bdd8-1490-426c-b5c9-aad3aa2deb6a-kube-api-access-mvnrm\") pod \"limitador-operator-controller-manager-6c549b6d85-vtn57\" (UID: \"c0c4bdd8-1490-426c-b5c9-aad3aa2deb6a\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtn57"
Sep 07 10:33:54.084270 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.084249    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d4c650e-31e8-427a-9178-4afa96b2a359-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-5wlsh\" (UID: \"1d4c650e-31e8-427a-9178-4afa96b2a359\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"
Sep 07 10:33:54.084396 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.084350    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3bad773-1e4c-41a6-a1f6-afd3074b3d18-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a3bad773-1e4c-41a6-a1f6-afd3074b3d18" (UID: "a3bad773-1e4c-41a6-a1f6-afd3074b3d18"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:33:54.084744 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.084723    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d4c650e-31e8-427a-9178-4afa96b2a359-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-5wlsh\" (UID: \"1d4c650e-31e8-427a-9178-4afa96b2a359\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"
Sep 07 10:33:54.086525 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.086501    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3bad773-1e4c-41a6-a1f6-afd3074b3d18-kube-api-access-4cwll" (OuterVolumeSpecName: "kube-api-access-4cwll") pod "a3bad773-1e4c-41a6-a1f6-afd3074b3d18" (UID: "a3bad773-1e4c-41a6-a1f6-afd3074b3d18"). InnerVolumeSpecName "kube-api-access-4cwll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:33:54.086676 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.086655    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d8fefac-f6f9-4794-b1cd-31984f60fa83-kube-api-access-zqqq8" (OuterVolumeSpecName: "kube-api-access-zqqq8") pod "1d8fefac-f6f9-4794-b1cd-31984f60fa83" (UID: "1d8fefac-f6f9-4794-b1cd-31984f60fa83"). InnerVolumeSpecName "kube-api-access-zqqq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:33:54.094601 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.094574    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvnrm\" (UniqueName: \"kubernetes.io/projected/c0c4bdd8-1490-426c-b5c9-aad3aa2deb6a-kube-api-access-mvnrm\") pod \"limitador-operator-controller-manager-6c549b6d85-vtn57\" (UID: \"c0c4bdd8-1490-426c-b5c9-aad3aa2deb6a\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtn57"
Sep 07 10:33:54.094902 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.094880    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hpch\" (UniqueName: \"kubernetes.io/projected/1d4c650e-31e8-427a-9178-4afa96b2a359-kube-api-access-6hpch\") pod \"kuadrant-operator-controller-manager-58fc56757c-5wlsh\" (UID: \"1d4c650e-31e8-427a-9178-4afa96b2a359\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"
Sep 07 10:33:54.185273 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.185197    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3bad773-1e4c-41a6-a1f6-afd3074b3d18-extensions-socket-volume\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:54.185273 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.185221    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqqq8\" (UniqueName: \"kubernetes.io/projected/1d8fefac-f6f9-4794-b1cd-31984f60fa83-kube-api-access-zqqq8\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:54.185273 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.185231    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cwll\" (UniqueName: \"kubernetes.io/projected/a3bad773-1e4c-41a6-a1f6-afd3074b3d18-kube-api-access-4cwll\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:54.216835 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.216799    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"
Sep 07 10:33:54.224117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.224096    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"
Sep 07 10:33:54.233262 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.233239    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtn57"
Sep 07 10:33:54.517265 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.517238    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"]
Sep 07 10:33:54.518863 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:33:54.518839    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d4c650e_31e8_427a_9178_4afa96b2a359.slice/crio-fd73029dc2aa1b460eed262844911ced3ca62a3d7bc9ef457c3725e68ff858a9 WatchSource:0}: Error finding container fd73029dc2aa1b460eed262844911ced3ca62a3d7bc9ef457c3725e68ff858a9: Status 404 returned error can't find the container with id fd73029dc2aa1b460eed262844911ced3ca62a3d7bc9ef457c3725e68ff858a9
Sep 07 10:33:54.519531 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.519507    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"]
Sep 07 10:33:54.519738 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:33:54.519721    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0ef1a5_fbda_423e_b224_3c28c1121a7d.slice/crio-395daf65ba25762dd6e57efdae1e3c20e02fc58d628a02313bea582f5be80fa5 WatchSource:0}: Error finding container 395daf65ba25762dd6e57efdae1e3c20e02fc58d628a02313bea582f5be80fa5: Status 404 returned error can't find the container with id 395daf65ba25762dd6e57efdae1e3c20e02fc58d628a02313bea582f5be80fa5
Sep 07 10:33:54.532713 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.532690    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtn57"]
Sep 07 10:33:54.533521 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:33:54.533497    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0c4bdd8_1490_426c_b5c9_aad3aa2deb6a.slice/crio-41070aaaee1299f81c6c9b65d861497f6769048c06f6cc828c55d28a01072513 WatchSource:0}: Error finding container 41070aaaee1299f81c6c9b65d861497f6769048c06f6cc828c55d28a01072513: Status 404 returned error can't find the container with id 41070aaaee1299f81c6c9b65d861497f6769048c06f6cc828c55d28a01072513
Sep 07 10:33:54.660933 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.660894    2577 generic.go:356] "Generic (PLEG): container finished" podID="1d8fefac-f6f9-4794-b1cd-31984f60fa83" containerID="204a1825583a57ea478c5447dfe3cabe8311a4c8e9bdda4632bba2047ee4dd03" exitCode=0
Sep 07 10:33:54.661408 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.660940    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:54.661408 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.661005    2577 scope.go:117] "RemoveContainer" containerID="204a1825583a57ea478c5447dfe3cabe8311a4c8e9bdda4632bba2047ee4dd03"
Sep 07 10:33:54.662961 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.662938    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtn57" event={"ID":"c0c4bdd8-1490-426c-b5c9-aad3aa2deb6a","Type":"ContainerStarted","Data":"ee257632b36fe6e42bdb63af95c8dacca5e8365439a89a196da3fcfb5f959076"}
Sep 07 10:33:54.663077 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.662988    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtn57" event={"ID":"c0c4bdd8-1490-426c-b5c9-aad3aa2deb6a","Type":"ContainerStarted","Data":"41070aaaee1299f81c6c9b65d861497f6769048c06f6cc828c55d28a01072513"}
Sep 07 10:33:54.663077 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.663028    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtn57"
Sep 07 10:33:54.663983 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:54.663952    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-zwkxl\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="1d8fefac-f6f9-4794-b1cd-31984f60fa83" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:54.664947 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.664924    2577 generic.go:356] "Generic (PLEG): container finished" podID="a3bad773-1e4c-41a6-a1f6-afd3074b3d18" containerID="f277e3be4cb453bfc9a11affa310c0053689bff716e7ccef75349a8cd0c75ccb" exitCode=0
Sep 07 10:33:54.665062 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.664958    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:54.666311 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:54.666279    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-nfpvb\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="a3bad773-1e4c-41a6-a1f6-afd3074b3d18" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:54.667614 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.667164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh" event={"ID":"1d4c650e-31e8-427a-9178-4afa96b2a359","Type":"ContainerStarted","Data":"9b1b82967c23f7be263fddd87d2c1c28834379930930ea3fb4835d496974aced"}
Sep 07 10:33:54.667614 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.667217    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh" event={"ID":"1d4c650e-31e8-427a-9178-4afa96b2a359","Type":"ContainerStarted","Data":"fd73029dc2aa1b460eed262844911ced3ca62a3d7bc9ef457c3725e68ff858a9"}
Sep 07 10:33:54.667614 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.667248    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"
Sep 07 10:33:54.668982 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:54.668957    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-nfpvb\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="a3bad773-1e4c-41a6-a1f6-afd3074b3d18" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:54.669305 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.669240    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f" event={"ID":"1a0ef1a5-fbda-423e-b224-3c28c1121a7d","Type":"ContainerStarted","Data":"b4d2980ce91f1a01e58e6fb2c9693510b838b2ad58314ce6632747ebc846b523"}
Sep 07 10:33:54.669305 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.669287    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"
Sep 07 10:33:54.669473 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.669292    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"
Sep 07 10:33:54.669473 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.669423    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f" event={"ID":"1a0ef1a5-fbda-423e-b224-3c28c1121a7d","Type":"ContainerStarted","Data":"395daf65ba25762dd6e57efdae1e3c20e02fc58d628a02313bea582f5be80fa5"}
Sep 07 10:33:54.671454 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.671434    2577 scope.go:117] "RemoveContainer" containerID="204a1825583a57ea478c5447dfe3cabe8311a4c8e9bdda4632bba2047ee4dd03"
Sep 07 10:33:54.671795 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:54.671769    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"204a1825583a57ea478c5447dfe3cabe8311a4c8e9bdda4632bba2047ee4dd03\": container with ID starting with 204a1825583a57ea478c5447dfe3cabe8311a4c8e9bdda4632bba2047ee4dd03 not found: ID does not exist" containerID="204a1825583a57ea478c5447dfe3cabe8311a4c8e9bdda4632bba2047ee4dd03"
Sep 07 10:33:54.671882 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.671805    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"204a1825583a57ea478c5447dfe3cabe8311a4c8e9bdda4632bba2047ee4dd03"} err="failed to get container status \"204a1825583a57ea478c5447dfe3cabe8311a4c8e9bdda4632bba2047ee4dd03\": rpc error: code = NotFound desc = could not find container \"204a1825583a57ea478c5447dfe3cabe8311a4c8e9bdda4632bba2047ee4dd03\": container with ID starting with 204a1825583a57ea478c5447dfe3cabe8311a4c8e9bdda4632bba2047ee4dd03 not found: ID does not exist"
Sep 07 10:33:54.671882 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.671829    2577 scope.go:117] "RemoveContainer" containerID="f277e3be4cb453bfc9a11affa310c0053689bff716e7ccef75349a8cd0c75ccb"
Sep 07 10:33:54.681016 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.680987    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"
Sep 07 10:33:54.684909 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.684886    2577 scope.go:117] "RemoveContainer" containerID="f277e3be4cb453bfc9a11affa310c0053689bff716e7ccef75349a8cd0c75ccb"
Sep 07 10:33:54.685263 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:54.685237    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f277e3be4cb453bfc9a11affa310c0053689bff716e7ccef75349a8cd0c75ccb\": container with ID starting with f277e3be4cb453bfc9a11affa310c0053689bff716e7ccef75349a8cd0c75ccb not found: ID does not exist" containerID="f277e3be4cb453bfc9a11affa310c0053689bff716e7ccef75349a8cd0c75ccb"
Sep 07 10:33:54.685349 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.685275    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f277e3be4cb453bfc9a11affa310c0053689bff716e7ccef75349a8cd0c75ccb"} err="failed to get container status \"f277e3be4cb453bfc9a11affa310c0053689bff716e7ccef75349a8cd0c75ccb\": rpc error: code = NotFound desc = could not find container \"f277e3be4cb453bfc9a11affa310c0053689bff716e7ccef75349a8cd0c75ccb\": container with ID starting with f277e3be4cb453bfc9a11affa310c0053689bff716e7ccef75349a8cd0c75ccb not found: ID does not exist"
Sep 07 10:33:54.690975 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.690928    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtn57" podStartSLOduration=1.690913766 podStartE2EDuration="1.690913766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:33:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:33:54.688854725 +0000 UTC m=+838.423028975" watchObservedRunningTime="2026-09-07 10:33:54.690913766 +0000 UTC m=+838.425088002"
Sep 07 10:33:54.691212 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:54.691190    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-zwkxl\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="1d8fefac-f6f9-4794-b1cd-31984f60fa83" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:54.694862 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:54.694836    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-zwkxl\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="1d8fefac-f6f9-4794-b1cd-31984f60fa83" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-zwkxl"
Sep 07 10:33:54.719386 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.719339    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh" podStartSLOduration=1.7193234739999999 podStartE2EDuration="1.719323474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:33:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:33:54.717695261 +0000 UTC m=+838.451869495" watchObservedRunningTime="2026-09-07 10:33:54.719323474 +0000 UTC m=+838.453497707"
Sep 07 10:33:54.721456 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:54.721429    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-nfpvb\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="a3bad773-1e4c-41a6-a1f6-afd3074b3d18" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-nfpvb"
Sep 07 10:33:54.752986 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.752899    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f" podStartSLOduration=1.7528833860000002 podStartE2EDuration="1.752883386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:33:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:33:54.750499313 +0000 UTC m=+838.484673546" watchObservedRunningTime="2026-09-07 10:33:54.752883386 +0000 UTC m=+838.487057619"
Sep 07 10:33:54.753706 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:54.753683    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-68cf589589-c6fvc\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="02b854d5-537d-4d3c-ab23-ec1a5c8b942b" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"
Sep 07 10:33:54.786950 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.786919    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d8fefac-f6f9-4794-b1cd-31984f60fa83" path="/var/lib/kubelet/pods/1d8fefac-f6f9-4794-b1cd-31984f60fa83/volumes"
Sep 07 10:33:54.787298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.787282    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3bad773-1e4c-41a6-a1f6-afd3074b3d18" path="/var/lib/kubelet/pods/a3bad773-1e4c-41a6-a1f6-afd3074b3d18/volumes"
Sep 07 10:33:54.789303 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.789275    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/02b854d5-537d-4d3c-ab23-ec1a5c8b942b-extensions-socket-volume\") pod \"02b854d5-537d-4d3c-ab23-ec1a5c8b942b\" (UID: \"02b854d5-537d-4d3c-ab23-ec1a5c8b942b\") "
Sep 07 10:33:54.789446 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.789372    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z4ss\" (UniqueName: \"kubernetes.io/projected/02b854d5-537d-4d3c-ab23-ec1a5c8b942b-kube-api-access-8z4ss\") pod \"02b854d5-537d-4d3c-ab23-ec1a5c8b942b\" (UID: \"02b854d5-537d-4d3c-ab23-ec1a5c8b942b\") "
Sep 07 10:33:54.789712 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.789662    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02b854d5-537d-4d3c-ab23-ec1a5c8b942b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "02b854d5-537d-4d3c-ab23-ec1a5c8b942b" (UID: "02b854d5-537d-4d3c-ab23-ec1a5c8b942b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:33:54.789809 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.789755    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/02b854d5-537d-4d3c-ab23-ec1a5c8b942b-extensions-socket-volume\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:54.791920 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.791892    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02b854d5-537d-4d3c-ab23-ec1a5c8b942b-kube-api-access-8z4ss" (OuterVolumeSpecName: "kube-api-access-8z4ss") pod "02b854d5-537d-4d3c-ab23-ec1a5c8b942b" (UID: "02b854d5-537d-4d3c-ab23-ec1a5c8b942b"). InnerVolumeSpecName "kube-api-access-8z4ss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:33:54.890383 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:54.890353    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z4ss\" (UniqueName: \"kubernetes.io/projected/02b854d5-537d-4d3c-ab23-ec1a5c8b942b-kube-api-access-8z4ss\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:33:55.599031 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:55.598990    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:55.680156 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:55.680097    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"
Sep 07 10:33:55.690279 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:55.690251    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-68cf589589-c6fvc\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="02b854d5-537d-4d3c-ab23-ec1a5c8b942b" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"
Sep 07 10:33:55.701982 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:55.701938    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:56.791685 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:56.791657    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02b854d5-537d-4d3c-ab23-ec1a5c8b942b" path="/var/lib/kubelet/pods/02b854d5-537d-4d3c-ab23-ec1a5c8b942b/volumes"
Sep 07 10:33:56.794211 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:33:56.794170    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-68cf589589-c6fvc\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="02b854d5-537d-4d3c-ab23-ec1a5c8b942b" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-c6fvc"
Sep 07 10:33:59.394680 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:59.394642    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:59.495400 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:59.495358    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:33:59.983698 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:33:59.983664    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:00.047750 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:00.047714    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:02.401274 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:02.401244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:02.501200 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:02.501162    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:05.682072 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:05.682043    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-vtn57"
Sep 07 10:34:05.682567 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:05.682228    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"
Sep 07 10:34:05.682567 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:05.682284    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"
Sep 07 10:34:05.778984 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:05.778951    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:05.800262 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:05.800222    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"]
Sep 07 10:34:05.800420 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:05.800397    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh" podUID="1d4c650e-31e8-427a-9178-4afa96b2a359" containerName="manager" containerID="cri-o://9b1b82967c23f7be263fddd87d2c1c28834379930930ea3fb4835d496974aced" gracePeriod=10
Sep 07 10:34:05.856743 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:05.856696    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:06.051230 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.051204    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"
Sep 07 10:34:06.068367 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.068331    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hpch\" (UniqueName: \"kubernetes.io/projected/1d4c650e-31e8-427a-9178-4afa96b2a359-kube-api-access-6hpch\") pod \"1d4c650e-31e8-427a-9178-4afa96b2a359\" (UID: \"1d4c650e-31e8-427a-9178-4afa96b2a359\") "
Sep 07 10:34:06.068561 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.068375    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d4c650e-31e8-427a-9178-4afa96b2a359-extensions-socket-volume\") pod \"1d4c650e-31e8-427a-9178-4afa96b2a359\" (UID: \"1d4c650e-31e8-427a-9178-4afa96b2a359\") "
Sep 07 10:34:06.068834 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.068804    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4c650e-31e8-427a-9178-4afa96b2a359-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1d4c650e-31e8-427a-9178-4afa96b2a359" (UID: "1d4c650e-31e8-427a-9178-4afa96b2a359"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:34:06.070552 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.070526    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d4c650e-31e8-427a-9178-4afa96b2a359-kube-api-access-6hpch" (OuterVolumeSpecName: "kube-api-access-6hpch") pod "1d4c650e-31e8-427a-9178-4afa96b2a359" (UID: "1d4c650e-31e8-427a-9178-4afa96b2a359"). InnerVolumeSpecName "kube-api-access-6hpch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:34:06.081859 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.081833    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"]
Sep 07 10:34:06.082296 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.082282    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4c650e-31e8-427a-9178-4afa96b2a359" containerName="manager"
Sep 07 10:34:06.082361 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.082298    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4c650e-31e8-427a-9178-4afa96b2a359" containerName="manager"
Sep 07 10:34:06.082361 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.082358    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4c650e-31e8-427a-9178-4afa96b2a359" containerName="manager"
Sep 07 10:34:06.085802 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.085786    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"
Sep 07 10:34:06.104699 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.104662    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"]
Sep 07 10:34:06.169525 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.169483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj8q2\" (UniqueName: \"kubernetes.io/projected/f3b4129a-d42c-4b66-b7c4-300dfb57d9d0-kube-api-access-wj8q2\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-t67xd\" (UID: \"f3b4129a-d42c-4b66-b7c4-300dfb57d9d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"
Sep 07 10:34:06.169708 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.169559    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3b4129a-d42c-4b66-b7c4-300dfb57d9d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-t67xd\" (UID: \"f3b4129a-d42c-4b66-b7c4-300dfb57d9d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"
Sep 07 10:34:06.169708 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.169592    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hpch\" (UniqueName: \"kubernetes.io/projected/1d4c650e-31e8-427a-9178-4afa96b2a359-kube-api-access-6hpch\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:34:06.169708 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.169601    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1d4c650e-31e8-427a-9178-4afa96b2a359-extensions-socket-volume\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:34:06.270267 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.270108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3b4129a-d42c-4b66-b7c4-300dfb57d9d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-t67xd\" (UID: \"f3b4129a-d42c-4b66-b7c4-300dfb57d9d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"
Sep 07 10:34:06.270267 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.270234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj8q2\" (UniqueName: \"kubernetes.io/projected/f3b4129a-d42c-4b66-b7c4-300dfb57d9d0-kube-api-access-wj8q2\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-t67xd\" (UID: \"f3b4129a-d42c-4b66-b7c4-300dfb57d9d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"
Sep 07 10:34:06.270540 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.270521    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3b4129a-d42c-4b66-b7c4-300dfb57d9d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-t67xd\" (UID: \"f3b4129a-d42c-4b66-b7c4-300dfb57d9d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"
Sep 07 10:34:06.288286 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.288252    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj8q2\" (UniqueName: \"kubernetes.io/projected/f3b4129a-d42c-4b66-b7c4-300dfb57d9d0-kube-api-access-wj8q2\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-t67xd\" (UID: \"f3b4129a-d42c-4b66-b7c4-300dfb57d9d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"
Sep 07 10:34:06.396290 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.396254    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"
Sep 07 10:34:06.679904 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.679881    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"]
Sep 07 10:34:06.682038 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:34:06.682014    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3b4129a_d42c_4b66_b7c4_300dfb57d9d0.slice/crio-8b14370fd639f8468405b12063911c265db523f89dc635751fb97925b9fe3e69 WatchSource:0}: Error finding container 8b14370fd639f8468405b12063911c265db523f89dc635751fb97925b9fe3e69: Status 404 returned error can't find the container with id 8b14370fd639f8468405b12063911c265db523f89dc635751fb97925b9fe3e69
Sep 07 10:34:06.752995 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.752962    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd" event={"ID":"f3b4129a-d42c-4b66-b7c4-300dfb57d9d0","Type":"ContainerStarted","Data":"8b14370fd639f8468405b12063911c265db523f89dc635751fb97925b9fe3e69"}
Sep 07 10:34:06.757979 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.757950    2577 generic.go:356] "Generic (PLEG): container finished" podID="1d4c650e-31e8-427a-9178-4afa96b2a359" containerID="9b1b82967c23f7be263fddd87d2c1c28834379930930ea3fb4835d496974aced" exitCode=0
Sep 07 10:34:06.758104 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.758019    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"
Sep 07 10:34:06.758104 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.758042    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh" event={"ID":"1d4c650e-31e8-427a-9178-4afa96b2a359","Type":"ContainerDied","Data":"9b1b82967c23f7be263fddd87d2c1c28834379930930ea3fb4835d496974aced"}
Sep 07 10:34:06.758104 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.758101    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh" event={"ID":"1d4c650e-31e8-427a-9178-4afa96b2a359","Type":"ContainerDied","Data":"fd73029dc2aa1b460eed262844911ced3ca62a3d7bc9ef457c3725e68ff858a9"}
Sep 07 10:34:06.758304 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.758123    2577 scope.go:117] "RemoveContainer" containerID="9b1b82967c23f7be263fddd87d2c1c28834379930930ea3fb4835d496974aced"
Sep 07 10:34:06.767455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.767436    2577 scope.go:117] "RemoveContainer" containerID="9b1b82967c23f7be263fddd87d2c1c28834379930930ea3fb4835d496974aced"
Sep 07 10:34:06.767730 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:34:06.767712    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b1b82967c23f7be263fddd87d2c1c28834379930930ea3fb4835d496974aced\": container with ID starting with 9b1b82967c23f7be263fddd87d2c1c28834379930930ea3fb4835d496974aced not found: ID does not exist" containerID="9b1b82967c23f7be263fddd87d2c1c28834379930930ea3fb4835d496974aced"
Sep 07 10:34:06.767794 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.767736    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b1b82967c23f7be263fddd87d2c1c28834379930930ea3fb4835d496974aced"} err="failed to get container status \"9b1b82967c23f7be263fddd87d2c1c28834379930930ea3fb4835d496974aced\": rpc error: code = NotFound desc = could not find container \"9b1b82967c23f7be263fddd87d2c1c28834379930930ea3fb4835d496974aced\": container with ID starting with 9b1b82967c23f7be263fddd87d2c1c28834379930930ea3fb4835d496974aced not found: ID does not exist"
Sep 07 10:34:06.788315 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.788279    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"]
Sep 07 10:34:06.797300 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:06.797273    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-5wlsh"]
Sep 07 10:34:07.765724 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:07.765684    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd" event={"ID":"f3b4129a-d42c-4b66-b7c4-300dfb57d9d0","Type":"ContainerStarted","Data":"1bf14678aba8b43a760032008911c67e5e7cc4c097dd9cfa047ff065e379c47d"}
Sep 07 10:34:07.766178 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:07.765757    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"
Sep 07 10:34:07.802349 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:07.802294    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd" podStartSLOduration=1.8022752720000002 podStartE2EDuration="1.802275272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:34:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:34:07.799907845 +0000 UTC m=+851.534082078" watchObservedRunningTime="2026-09-07 10:34:07.802275272 +0000 UTC m=+851.536449516"
Sep 07 10:34:08.409242 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:08.409206    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:08.505550 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:08.505512    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:08.605113 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:08.605070    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:08.784334 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:08.784299    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d4c650e-31e8-427a-9178-4afa96b2a359" path="/var/lib/kubelet/pods/1d4c650e-31e8-427a-9178-4afa96b2a359/volumes"
Sep 07 10:34:15.437550 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:15.437511    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:15.535684 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:15.535648    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:18.775440 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:18.775410    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"
Sep 07 10:34:18.833514 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:18.833485    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"]
Sep 07 10:34:18.833749 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:18.833724    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f" podUID="1a0ef1a5-fbda-423e-b224-3c28c1121a7d" containerName="manager" containerID="cri-o://b4d2980ce91f1a01e58e6fb2c9693510b838b2ad58314ce6632747ebc846b523" gracePeriod=10
Sep 07 10:34:19.093711 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.093686    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"
Sep 07 10:34:19.171473 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.171448    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fszws\" (UniqueName: \"kubernetes.io/projected/1a0ef1a5-fbda-423e-b224-3c28c1121a7d-kube-api-access-fszws\") pod \"1a0ef1a5-fbda-423e-b224-3c28c1121a7d\" (UID: \"1a0ef1a5-fbda-423e-b224-3c28c1121a7d\") "
Sep 07 10:34:19.171613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.171500    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a0ef1a5-fbda-423e-b224-3c28c1121a7d-extensions-socket-volume\") pod \"1a0ef1a5-fbda-423e-b224-3c28c1121a7d\" (UID: \"1a0ef1a5-fbda-423e-b224-3c28c1121a7d\") "
Sep 07 10:34:19.171912 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.171891    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a0ef1a5-fbda-423e-b224-3c28c1121a7d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1a0ef1a5-fbda-423e-b224-3c28c1121a7d" (UID: "1a0ef1a5-fbda-423e-b224-3c28c1121a7d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:34:19.173697 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.173669    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a0ef1a5-fbda-423e-b224-3c28c1121a7d-kube-api-access-fszws" (OuterVolumeSpecName: "kube-api-access-fszws") pod "1a0ef1a5-fbda-423e-b224-3c28c1121a7d" (UID: "1a0ef1a5-fbda-423e-b224-3c28c1121a7d"). InnerVolumeSpecName "kube-api-access-fszws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:34:19.272528 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.272507    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fszws\" (UniqueName: \"kubernetes.io/projected/1a0ef1a5-fbda-423e-b224-3c28c1121a7d-kube-api-access-fszws\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:34:19.272528 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.272529    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a0ef1a5-fbda-423e-b224-3c28c1121a7d-extensions-socket-volume\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:34:19.837524 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.837490    2577 generic.go:356] "Generic (PLEG): container finished" podID="1a0ef1a5-fbda-423e-b224-3c28c1121a7d" containerID="b4d2980ce91f1a01e58e6fb2c9693510b838b2ad58314ce6632747ebc846b523" exitCode=0
Sep 07 10:34:19.837916 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.837538    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"
Sep 07 10:34:19.837916 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.837567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f" event={"ID":"1a0ef1a5-fbda-423e-b224-3c28c1121a7d","Type":"ContainerDied","Data":"b4d2980ce91f1a01e58e6fb2c9693510b838b2ad58314ce6632747ebc846b523"}
Sep 07 10:34:19.837916 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.837628    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f" event={"ID":"1a0ef1a5-fbda-423e-b224-3c28c1121a7d","Type":"ContainerDied","Data":"395daf65ba25762dd6e57efdae1e3c20e02fc58d628a02313bea582f5be80fa5"}
Sep 07 10:34:19.837916 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.837657    2577 scope.go:117] "RemoveContainer" containerID="b4d2980ce91f1a01e58e6fb2c9693510b838b2ad58314ce6632747ebc846b523"
Sep 07 10:34:19.846922 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.846905    2577 scope.go:117] "RemoveContainer" containerID="b4d2980ce91f1a01e58e6fb2c9693510b838b2ad58314ce6632747ebc846b523"
Sep 07 10:34:19.847268 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:34:19.847248    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4d2980ce91f1a01e58e6fb2c9693510b838b2ad58314ce6632747ebc846b523\": container with ID starting with b4d2980ce91f1a01e58e6fb2c9693510b838b2ad58314ce6632747ebc846b523 not found: ID does not exist" containerID="b4d2980ce91f1a01e58e6fb2c9693510b838b2ad58314ce6632747ebc846b523"
Sep 07 10:34:19.847347 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.847278    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4d2980ce91f1a01e58e6fb2c9693510b838b2ad58314ce6632747ebc846b523"} err="failed to get container status \"b4d2980ce91f1a01e58e6fb2c9693510b838b2ad58314ce6632747ebc846b523\": rpc error: code = NotFound desc = could not find container \"b4d2980ce91f1a01e58e6fb2c9693510b838b2ad58314ce6632747ebc846b523\": container with ID starting with b4d2980ce91f1a01e58e6fb2c9693510b838b2ad58314ce6632747ebc846b523 not found: ID does not exist"
Sep 07 10:34:19.869542 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.869519    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"]
Sep 07 10:34:19.874565 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:19.874541    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-qzc7f"]
Sep 07 10:34:20.784323 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:20.784289    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a0ef1a5-fbda-423e-b224-3c28c1121a7d" path="/var/lib/kubelet/pods/1a0ef1a5-fbda-423e-b224-3c28c1121a7d/volumes"
Sep 07 10:34:22.261875 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:22.261836    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:22.565463 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:22.565367    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:38.538528 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.538474    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:38.543371 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.543342    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"]
Sep 07 10:34:38.543795 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.543778    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a0ef1a5-fbda-423e-b224-3c28c1121a7d" containerName="manager"
Sep 07 10:34:38.543857 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.543798    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a0ef1a5-fbda-423e-b224-3c28c1121a7d" containerName="manager"
Sep 07 10:34:38.543857 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.543855    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a0ef1a5-fbda-423e-b224-3c28c1121a7d" containerName="manager"
Sep 07 10:34:38.547100 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.547085    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"
Sep 07 10:34:38.549773 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.549756    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 07 10:34:38.554820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.554800    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"]
Sep 07 10:34:38.632174 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.632123    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f58f12d9-1a30-4316-926f-8d88c5a4fb82-config-file\") pod \"limitador-limitador-859c4d96d6-z7rsp\" (UID: \"f58f12d9-1a30-4316-926f-8d88c5a4fb82\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"
Sep 07 10:34:38.632334 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.632284    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrcz6\" (UniqueName: \"kubernetes.io/projected/f58f12d9-1a30-4316-926f-8d88c5a4fb82-kube-api-access-lrcz6\") pod \"limitador-limitador-859c4d96d6-z7rsp\" (UID: \"f58f12d9-1a30-4316-926f-8d88c5a4fb82\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"
Sep 07 10:34:38.646272 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.646237    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"]
Sep 07 10:34:38.649425 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.649396    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:38.733566 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.733535    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrcz6\" (UniqueName: \"kubernetes.io/projected/f58f12d9-1a30-4316-926f-8d88c5a4fb82-kube-api-access-lrcz6\") pod \"limitador-limitador-859c4d96d6-z7rsp\" (UID: \"f58f12d9-1a30-4316-926f-8d88c5a4fb82\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"
Sep 07 10:34:38.733696 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.733577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f58f12d9-1a30-4316-926f-8d88c5a4fb82-config-file\") pod \"limitador-limitador-859c4d96d6-z7rsp\" (UID: \"f58f12d9-1a30-4316-926f-8d88c5a4fb82\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"
Sep 07 10:34:38.734167 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.734121    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f58f12d9-1a30-4316-926f-8d88c5a4fb82-config-file\") pod \"limitador-limitador-859c4d96d6-z7rsp\" (UID: \"f58f12d9-1a30-4316-926f-8d88c5a4fb82\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"
Sep 07 10:34:38.744094 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.744068    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrcz6\" (UniqueName: \"kubernetes.io/projected/f58f12d9-1a30-4316-926f-8d88c5a4fb82-kube-api-access-lrcz6\") pod \"limitador-limitador-859c4d96d6-z7rsp\" (UID: \"f58f12d9-1a30-4316-926f-8d88c5a4fb82\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"
Sep 07 10:34:38.749846 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.749816    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:38.859269 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.859189    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"
Sep 07 10:34:38.994551 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.994517    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:34:38.999266 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:38.999245    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"
Sep 07 10:34:39.006458 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:39.006430    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:34:39.048946 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:39.048919    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:34:39.136093 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:39.136051    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b7x8\" (UniqueName: \"kubernetes.io/projected/ef69e096-39ae-40c6-8124-1e23cc1cd220-kube-api-access-7b7x8\") pod \"limitador-limitador-85d9db4d5d-4qzwx\" (UID: \"ef69e096-39ae-40c6-8124-1e23cc1cd220\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"
Sep 07 10:34:39.136205 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:39.136115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ef69e096-39ae-40c6-8124-1e23cc1cd220-config-file\") pod \"limitador-limitador-85d9db4d5d-4qzwx\" (UID: \"ef69e096-39ae-40c6-8124-1e23cc1cd220\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"
Sep 07 10:34:39.136943 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:39.136924    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"]
Sep 07 10:34:39.139024 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:34:39.138998    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf58f12d9_1a30_4316_926f_8d88c5a4fb82.slice/crio-8a09f11f97d5008ab08782929e042f9e72b82a2b37e3dd021a5303c8562f0412 WatchSource:0}: Error finding container 8a09f11f97d5008ab08782929e042f9e72b82a2b37e3dd021a5303c8562f0412: Status 404 returned error can't find the container with id 8a09f11f97d5008ab08782929e042f9e72b82a2b37e3dd021a5303c8562f0412
Sep 07 10:34:39.237439 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:39.237406    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b7x8\" (UniqueName: \"kubernetes.io/projected/ef69e096-39ae-40c6-8124-1e23cc1cd220-kube-api-access-7b7x8\") pod \"limitador-limitador-85d9db4d5d-4qzwx\" (UID: \"ef69e096-39ae-40c6-8124-1e23cc1cd220\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"
Sep 07 10:34:39.237439 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:39.237450    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ef69e096-39ae-40c6-8124-1e23cc1cd220-config-file\") pod \"limitador-limitador-85d9db4d5d-4qzwx\" (UID: \"ef69e096-39ae-40c6-8124-1e23cc1cd220\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"
Sep 07 10:34:39.238012 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:39.237994    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ef69e096-39ae-40c6-8124-1e23cc1cd220-config-file\") pod \"limitador-limitador-85d9db4d5d-4qzwx\" (UID: \"ef69e096-39ae-40c6-8124-1e23cc1cd220\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"
Sep 07 10:34:39.246604 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:39.246576    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b7x8\" (UniqueName: \"kubernetes.io/projected/ef69e096-39ae-40c6-8124-1e23cc1cd220-kube-api-access-7b7x8\") pod \"limitador-limitador-85d9db4d5d-4qzwx\" (UID: \"ef69e096-39ae-40c6-8124-1e23cc1cd220\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"
Sep 07 10:34:39.311173 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:39.311122    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"
Sep 07 10:34:39.391774 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:39.391703    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:39.487014 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:39.486971    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:39.592983 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:39.592955    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:34:39.594476 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:34:39.594452    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef69e096_39ae_40c6_8124_1e23cc1cd220.slice/crio-d16046e79ec653e5087bea3aed6bef6f2094687a2b31d53a9ce2afd4e5b2bf30 WatchSource:0}: Error finding container d16046e79ec653e5087bea3aed6bef6f2094687a2b31d53a9ce2afd4e5b2bf30: Status 404 returned error can't find the container with id d16046e79ec653e5087bea3aed6bef6f2094687a2b31d53a9ce2afd4e5b2bf30
Sep 07 10:34:39.951164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:39.951111    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp" event={"ID":"f58f12d9-1a30-4316-926f-8d88c5a4fb82","Type":"ContainerStarted","Data":"8a09f11f97d5008ab08782929e042f9e72b82a2b37e3dd021a5303c8562f0412"}
Sep 07 10:34:39.952282 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:39.952258    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx" event={"ID":"ef69e096-39ae-40c6-8124-1e23cc1cd220","Type":"ContainerStarted","Data":"d16046e79ec653e5087bea3aed6bef6f2094687a2b31d53a9ce2afd4e5b2bf30"}
Sep 07 10:34:40.959493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:40.959464    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx" event={"ID":"ef69e096-39ae-40c6-8124-1e23cc1cd220","Type":"ContainerStarted","Data":"8f705d7e812c53402f47a222e4b584cbfa5aec8d55fd54bc55312ff87526edd3"}
Sep 07 10:34:40.959927 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:40.959557    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"
Sep 07 10:34:40.978269 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:40.978211    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx" podStartSLOduration=2.059353748 podStartE2EDuration="2.978193914s" totalImagesPullingTime="918.840166ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:34:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:34:40.977165971 +0000 UTC m=+884.711340222" watchObservedRunningTime="2026-09-07 10:34:40.978193914 +0000 UTC m=+884.712368147"
Sep 07 10:34:41.966569 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:41.966528    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp" event={"ID":"f58f12d9-1a30-4316-926f-8d88c5a4fb82","Type":"ContainerStarted","Data":"6248065006cb23616f8d38bf3f2c167d144fca33f0bfbc4d9e4846fd436df8ef"}
Sep 07 10:34:41.966992 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:41.966814    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"
Sep 07 10:34:42.005387 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:42.005334    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp" podStartSLOduration=1.846272635 podStartE2EDuration="4.005317975s" totalImagesPullingTime="2.15904534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:34:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:34:42.003500728 +0000 UTC m=+885.737674978" watchObservedRunningTime="2026-09-07 10:34:42.005317975 +0000 UTC m=+885.739492208"
Sep 07 10:34:51.968163 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:51.968049    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"
Sep 07 10:34:52.020526 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:52.020486    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"]
Sep 07 10:34:52.020726 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:52.020697    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp" podUID="f58f12d9-1a30-4316-926f-8d88c5a4fb82" containerName="limitador" containerID="cri-o://6248065006cb23616f8d38bf3f2c167d144fca33f0bfbc4d9e4846fd436df8ef" gracePeriod=30
Sep 07 10:34:52.021420 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:52.021324    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"
Sep 07 10:34:52.572428 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:52.572399    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"
Sep 07 10:34:52.650230 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:52.650094    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrcz6\" (UniqueName: \"kubernetes.io/projected/f58f12d9-1a30-4316-926f-8d88c5a4fb82-kube-api-access-lrcz6\") pod \"f58f12d9-1a30-4316-926f-8d88c5a4fb82\" (UID: \"f58f12d9-1a30-4316-926f-8d88c5a4fb82\") "
Sep 07 10:34:52.650230 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:52.650177    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f58f12d9-1a30-4316-926f-8d88c5a4fb82-config-file\") pod \"f58f12d9-1a30-4316-926f-8d88c5a4fb82\" (UID: \"f58f12d9-1a30-4316-926f-8d88c5a4fb82\") "
Sep 07 10:34:52.650503 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:52.650476    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f58f12d9-1a30-4316-926f-8d88c5a4fb82-config-file" (OuterVolumeSpecName: "config-file") pod "f58f12d9-1a30-4316-926f-8d88c5a4fb82" (UID: "f58f12d9-1a30-4316-926f-8d88c5a4fb82"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:34:52.652559 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:52.652532    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f58f12d9-1a30-4316-926f-8d88c5a4fb82-kube-api-access-lrcz6" (OuterVolumeSpecName: "kube-api-access-lrcz6") pod "f58f12d9-1a30-4316-926f-8d88c5a4fb82" (UID: "f58f12d9-1a30-4316-926f-8d88c5a4fb82"). InnerVolumeSpecName "kube-api-access-lrcz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:34:52.751455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:52.751414    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrcz6\" (UniqueName: \"kubernetes.io/projected/f58f12d9-1a30-4316-926f-8d88c5a4fb82-kube-api-access-lrcz6\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:34:52.751455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:52.751449    2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f58f12d9-1a30-4316-926f-8d88c5a4fb82-config-file\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:34:53.032596 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:53.032560    2577 generic.go:356] "Generic (PLEG): container finished" podID="f58f12d9-1a30-4316-926f-8d88c5a4fb82" containerID="6248065006cb23616f8d38bf3f2c167d144fca33f0bfbc4d9e4846fd436df8ef" exitCode=0
Sep 07 10:34:53.032596 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:53.032597    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"
Sep 07 10:34:53.033101 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:53.032641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp" event={"ID":"f58f12d9-1a30-4316-926f-8d88c5a4fb82","Type":"ContainerDied","Data":"6248065006cb23616f8d38bf3f2c167d144fca33f0bfbc4d9e4846fd436df8ef"}
Sep 07 10:34:53.033101 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:53.032696    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-z7rsp" event={"ID":"f58f12d9-1a30-4316-926f-8d88c5a4fb82","Type":"ContainerDied","Data":"8a09f11f97d5008ab08782929e042f9e72b82a2b37e3dd021a5303c8562f0412"}
Sep 07 10:34:53.033101 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:53.032719    2577 scope.go:117] "RemoveContainer" containerID="6248065006cb23616f8d38bf3f2c167d144fca33f0bfbc4d9e4846fd436df8ef"
Sep 07 10:34:53.041756 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:53.041739    2577 scope.go:117] "RemoveContainer" containerID="6248065006cb23616f8d38bf3f2c167d144fca33f0bfbc4d9e4846fd436df8ef"
Sep 07 10:34:53.042047 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:34:53.042029    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6248065006cb23616f8d38bf3f2c167d144fca33f0bfbc4d9e4846fd436df8ef\": container with ID starting with 6248065006cb23616f8d38bf3f2c167d144fca33f0bfbc4d9e4846fd436df8ef not found: ID does not exist" containerID="6248065006cb23616f8d38bf3f2c167d144fca33f0bfbc4d9e4846fd436df8ef"
Sep 07 10:34:53.042099 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:53.042058    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6248065006cb23616f8d38bf3f2c167d144fca33f0bfbc4d9e4846fd436df8ef"} err="failed to get container status \"6248065006cb23616f8d38bf3f2c167d144fca33f0bfbc4d9e4846fd436df8ef\": rpc error: code = NotFound desc = could not find container \"6248065006cb23616f8d38bf3f2c167d144fca33f0bfbc4d9e4846fd436df8ef\": container with ID starting with 6248065006cb23616f8d38bf3f2c167d144fca33f0bfbc4d9e4846fd436df8ef not found: ID does not exist"
Sep 07 10:34:53.052832 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:53.052808    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"]
Sep 07 10:34:53.057721 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:53.057697    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-z7rsp"]
Sep 07 10:34:54.785045 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:54.785013    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f58f12d9-1a30-4316-926f-8d88c5a4fb82" path="/var/lib/kubelet/pods/f58f12d9-1a30-4316-926f-8d88c5a4fb82/volumes"
Sep 07 10:34:55.739112 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:55.739047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:55.839630 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:55.839593    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:56.118761 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:56.118681    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:56.220345 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:56.220311    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:56.624839 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:56.624806    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:34:56.749924 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:56.749899    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:34:56.762623 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:34:56.762601    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:35:02.794497 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:02.794457    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:08.872588 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:08.872520    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:09.066733 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.066695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:09.111053 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.111021    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-gvg4h"]
Sep 07 10:35:09.111436 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.111422    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f58f12d9-1a30-4316-926f-8d88c5a4fb82" containerName="limitador"
Sep 07 10:35:09.111491 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.111437    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f58f12d9-1a30-4316-926f-8d88c5a4fb82" containerName="limitador"
Sep 07 10:35:09.111526 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.111496    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f58f12d9-1a30-4316-926f-8d88c5a4fb82" containerName="limitador"
Sep 07 10:35:09.116107 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.116089    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-gvg4h"
Sep 07 10:35:09.120012 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.119985    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4tp6s\""
Sep 07 10:35:09.124231 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.124167    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-gvg4h"]
Sep 07 10:35:09.197151 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.197094    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn44f\" (UniqueName: \"kubernetes.io/projected/fd4affe3-1d11-489f-bf0a-d2c76b77a6b6-kube-api-access-rn44f\") pod \"authorino-5dc7ccb5f-gvg4h\" (UID: \"fd4affe3-1d11-489f-bf0a-d2c76b77a6b6\") " pod="kuadrant-system/authorino-5dc7ccb5f-gvg4h"
Sep 07 10:35:09.245253 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.245219    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-gvg4h"]
Sep 07 10:35:09.245519 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:35:09.245500    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rn44f], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5dc7ccb5f-gvg4h" podUID="fd4affe3-1d11-489f-bf0a-d2c76b77a6b6"
Sep 07 10:35:09.269470 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.269438    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-687bd45585-qt675"]
Sep 07 10:35:09.272836 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.272816    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-qt675"
Sep 07 10:35:09.275422 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.275406    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 07 10:35:09.280078 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.279994    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-687bd45585-qt675"]
Sep 07 10:35:09.298412 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.298384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn44f\" (UniqueName: \"kubernetes.io/projected/fd4affe3-1d11-489f-bf0a-d2c76b77a6b6-kube-api-access-rn44f\") pod \"authorino-5dc7ccb5f-gvg4h\" (UID: \"fd4affe3-1d11-489f-bf0a-d2c76b77a6b6\") " pod="kuadrant-system/authorino-5dc7ccb5f-gvg4h"
Sep 07 10:35:09.298533 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.298437    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trqjp\" (UniqueName: \"kubernetes.io/projected/ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7-kube-api-access-trqjp\") pod \"authorino-687bd45585-qt675\" (UID: \"ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7\") " pod="kuadrant-system/authorino-687bd45585-qt675"
Sep 07 10:35:09.298533 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.298488    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7-tls-cert\") pod \"authorino-687bd45585-qt675\" (UID: \"ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7\") " pod="kuadrant-system/authorino-687bd45585-qt675"
Sep 07 10:35:09.312102 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.312079    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn44f\" (UniqueName: \"kubernetes.io/projected/fd4affe3-1d11-489f-bf0a-d2c76b77a6b6-kube-api-access-rn44f\") pod \"authorino-5dc7ccb5f-gvg4h\" (UID: \"fd4affe3-1d11-489f-bf0a-d2c76b77a6b6\") " pod="kuadrant-system/authorino-5dc7ccb5f-gvg4h"
Sep 07 10:35:09.339034 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.339005    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-qt675"]
Sep 07 10:35:09.339309 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:35:09.339288    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-trqjp tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-687bd45585-qt675" podUID="ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7"
Sep 07 10:35:09.399622 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.399541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trqjp\" (UniqueName: \"kubernetes.io/projected/ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7-kube-api-access-trqjp\") pod \"authorino-687bd45585-qt675\" (UID: \"ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7\") " pod="kuadrant-system/authorino-687bd45585-qt675"
Sep 07 10:35:09.399778 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.399642    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7-tls-cert\") pod \"authorino-687bd45585-qt675\" (UID: \"ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7\") " pod="kuadrant-system/authorino-687bd45585-qt675"
Sep 07 10:35:09.402200 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.402174    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7-tls-cert\") pod \"authorino-687bd45585-qt675\" (UID: \"ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7\") " pod="kuadrant-system/authorino-687bd45585-qt675"
Sep 07 10:35:09.408599 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:09.408569    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trqjp\" (UniqueName: \"kubernetes.io/projected/ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7-kube-api-access-trqjp\") pod \"authorino-687bd45585-qt675\" (UID: \"ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7\") " pod="kuadrant-system/authorino-687bd45585-qt675"
Sep 07 10:35:10.130414 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:10.130381    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-qt675"
Sep 07 10:35:10.130844 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:10.130381    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-gvg4h"
Sep 07 10:35:10.137836 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:10.137812    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-qt675"
Sep 07 10:35:10.140505 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:10.140490    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-gvg4h"
Sep 07 10:35:10.204785 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:10.204749    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7-tls-cert\") pod \"ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7\" (UID: \"ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7\") "
Sep 07 10:35:10.204785 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:10.204791    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trqjp\" (UniqueName: \"kubernetes.io/projected/ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7-kube-api-access-trqjp\") pod \"ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7\" (UID: \"ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7\") "
Sep 07 10:35:10.205025 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:10.204861    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rn44f\" (UniqueName: \"kubernetes.io/projected/fd4affe3-1d11-489f-bf0a-d2c76b77a6b6-kube-api-access-rn44f\") pod \"fd4affe3-1d11-489f-bf0a-d2c76b77a6b6\" (UID: \"fd4affe3-1d11-489f-bf0a-d2c76b77a6b6\") "
Sep 07 10:35:10.207112 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:10.207079    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd4affe3-1d11-489f-bf0a-d2c76b77a6b6-kube-api-access-rn44f" (OuterVolumeSpecName: "kube-api-access-rn44f") pod "fd4affe3-1d11-489f-bf0a-d2c76b77a6b6" (UID: "fd4affe3-1d11-489f-bf0a-d2c76b77a6b6"). InnerVolumeSpecName "kube-api-access-rn44f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:35:10.207242 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:10.207150    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7" (UID: "ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:35:10.207287 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:10.207244    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7-kube-api-access-trqjp" (OuterVolumeSpecName: "kube-api-access-trqjp") pod "ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7" (UID: "ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7"). InnerVolumeSpecName "kube-api-access-trqjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:35:10.305864 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:10.305829    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7-tls-cert\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:35:10.305864 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:10.305855    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trqjp\" (UniqueName: \"kubernetes.io/projected/ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7-kube-api-access-trqjp\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:35:10.305864 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:10.305867    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rn44f\" (UniqueName: \"kubernetes.io/projected/fd4affe3-1d11-489f-bf0a-d2c76b77a6b6-kube-api-access-rn44f\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:35:11.135700 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:11.135624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-qt675"
Sep 07 10:35:11.136067 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:11.135626    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-gvg4h"
Sep 07 10:35:11.173723 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:11.173689    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-qt675"]
Sep 07 10:35:11.175759 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:11.175737    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-687bd45585-qt675"]
Sep 07 10:35:11.198652 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:11.198626    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-gvg4h"]
Sep 07 10:35:11.202854 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:11.202833    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-gvg4h"]
Sep 07 10:35:12.784762 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:12.784721    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7" path="/var/lib/kubelet/pods/ad2e1adb-63f9-4e33-80ee-81f31f5cb9b7/volumes"
Sep 07 10:35:12.785271 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:12.785044    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd4affe3-1d11-489f-bf0a-d2c76b77a6b6" path="/var/lib/kubelet/pods/fd4affe3-1d11-489f-bf0a-d2c76b77a6b6/volumes"
Sep 07 10:35:16.457795 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:16.457760    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:16.557393 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:16.557354    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:16.618622 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:16.618584    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:16.723150 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:16.723042    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:17.056454 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:17.056356    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:17.833037 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:17.832996    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:20.617968 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:20.617930    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:20.722323 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:20.722289    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:52.519717 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:52.519678    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:54.179630 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.179596    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"]
Sep 07 10:35:54.183100 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.183074    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:54.188386 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.188366    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 07 10:35:54.189911 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.189552    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 07 10:35:54.189911 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.189579    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 07 10:35:54.190252 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.190236    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-bgjkj\""
Sep 07 10:35:54.190252 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.190248    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 07 10:35:54.202040 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.202018    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"]
Sep 07 10:35:54.209702 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.209672    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/469c398d-a67f-4a35-8e1e-32ae679ca5ca-openshift-service-ca\") pod \"maas-api-58ff47f946-pxd4s\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") " pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:54.209841 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.209712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/469c398d-a67f-4a35-8e1e-32ae679ca5ca-metrics-certs\") pod \"maas-api-58ff47f946-pxd4s\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") " pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:54.209841 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.209750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/469c398d-a67f-4a35-8e1e-32ae679ca5ca-maas-api-tls\") pod \"maas-api-58ff47f946-pxd4s\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") " pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:54.209841 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.209817    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjxzn\" (UniqueName: \"kubernetes.io/projected/469c398d-a67f-4a35-8e1e-32ae679ca5ca-kube-api-access-sjxzn\") pod \"maas-api-58ff47f946-pxd4s\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") " pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:54.225460 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.225432    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:54.310610 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.310583    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjxzn\" (UniqueName: \"kubernetes.io/projected/469c398d-a67f-4a35-8e1e-32ae679ca5ca-kube-api-access-sjxzn\") pod \"maas-api-58ff47f946-pxd4s\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") " pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:54.310760 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.310629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/469c398d-a67f-4a35-8e1e-32ae679ca5ca-openshift-service-ca\") pod \"maas-api-58ff47f946-pxd4s\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") " pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:54.310760 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.310682    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/469c398d-a67f-4a35-8e1e-32ae679ca5ca-metrics-certs\") pod \"maas-api-58ff47f946-pxd4s\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") " pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:54.310760 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.310739    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/469c398d-a67f-4a35-8e1e-32ae679ca5ca-maas-api-tls\") pod \"maas-api-58ff47f946-pxd4s\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") " pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:54.310868 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:35:54.310840    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 07 10:35:54.310913 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:35:54.310908    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/469c398d-a67f-4a35-8e1e-32ae679ca5ca-maas-api-tls podName:469c398d-a67f-4a35-8e1e-32ae679ca5ca nodeName:}" failed. No retries permitted until 2026-09-07 10:35:54.810885155 +0000 UTC m=+958.545059381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/469c398d-a67f-4a35-8e1e-32ae679ca5ca-maas-api-tls") pod "maas-api-58ff47f946-pxd4s" (UID: "469c398d-a67f-4a35-8e1e-32ae679ca5ca") : secret "maas-api-serving-cert" not found
Sep 07 10:35:54.311602 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.311513    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/469c398d-a67f-4a35-8e1e-32ae679ca5ca-openshift-service-ca\") pod \"maas-api-58ff47f946-pxd4s\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") " pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:54.313461 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.313439    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/469c398d-a67f-4a35-8e1e-32ae679ca5ca-metrics-certs\") pod \"maas-api-58ff47f946-pxd4s\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") " pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:54.319498 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.319474    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjxzn\" (UniqueName: \"kubernetes.io/projected/469c398d-a67f-4a35-8e1e-32ae679ca5ca-kube-api-access-sjxzn\") pod \"maas-api-58ff47f946-pxd4s\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") " pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:54.325072 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.325022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:54.426272 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.426238    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:54.597026 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.596991    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:54.697286 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.697256    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:35:54.814965 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.814934    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/469c398d-a67f-4a35-8e1e-32ae679ca5ca-maas-api-tls\") pod \"maas-api-58ff47f946-pxd4s\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") " pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:54.817679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.817654    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/469c398d-a67f-4a35-8e1e-32ae679ca5ca-maas-api-tls\") pod \"maas-api-58ff47f946-pxd4s\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") " pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:54.864960 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:54.864896    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:35:55.094004 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:55.093973    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:55.370085 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:55.370061    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"]
Sep 07 10:35:55.371555 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:35:55.371534    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod469c398d_a67f_4a35_8e1e_32ae679ca5ca.slice/crio-8ee11416a03abec30772f49f9e07eeda5ede3533f1f46df57e49dbf5697adf90 WatchSource:0}: Error finding container 8ee11416a03abec30772f49f9e07eeda5ede3533f1f46df57e49dbf5697adf90: Status 404 returned error can't find the container with id 8ee11416a03abec30772f49f9e07eeda5ede3533f1f46df57e49dbf5697adf90
Sep 07 10:35:55.372960 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:55.372943    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 10:35:55.384970 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:55.384940    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s" event={"ID":"469c398d-a67f-4a35-8e1e-32ae679ca5ca","Type":"ContainerStarted","Data":"8ee11416a03abec30772f49f9e07eeda5ede3533f1f46df57e49dbf5697adf90"}
Sep 07 10:35:58.410486 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:58.410452    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s" event={"ID":"469c398d-a67f-4a35-8e1e-32ae679ca5ca","Type":"ContainerStarted","Data":"8f868d5d4571defc3566504d15749e1af3e2ce005450a8596385516aadf48eb7"}
Sep 07 10:35:58.410833 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:58.410591    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:35:58.427552 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:58.427509    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s" podStartSLOduration=1.566076139 podStartE2EDuration="4.427496013s" totalImagesPullingTime="2.861419874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:35:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:35:58.425905758 +0000 UTC m=+962.160079991" watchObservedRunningTime="2026-09-07 10:35:58.427496013 +0000 UTC m=+962.161670253"
Sep 07 10:35:59.125657 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:35:59.125619    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:36:04.421504 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:04.421472    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 10:36:05.448892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:05.448858    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:36:05.557200 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:05.557170    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:36:14.357267 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:14.357228    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:36:14.841245 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:14.841215    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:36:24.702502 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:24.702414    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:36:24.762571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:24.762545    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:36:26.859793 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:26.859761    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:36:27.727515 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:27.727481    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:36:36.428929 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.428895    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"]
Sep 07 10:36:36.433288 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.433266    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.435885 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.435867    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 07 10:36:36.436894 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.436874    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 07 10:36:36.437001 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.436922    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 07 10:36:36.437001 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.436983    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-v8l9q\""
Sep 07 10:36:36.442245 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.442224    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"]
Sep 07 10:36:36.454472 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.454443    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:36:36.557422 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.557389    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.557589 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.557432    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.557589 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.557450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.557589 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.557535    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.557589 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.557571    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.557752 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.557596    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vx9b\" (UniqueName: \"kubernetes.io/projected/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-kube-api-access-7vx9b\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.658378 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.658348    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.658378 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.658386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vx9b\" (UniqueName: \"kubernetes.io/projected/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-kube-api-access-7vx9b\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.658607 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.658437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.658607 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.658463    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.658607 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.658476    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.658607 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.658505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.658838 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.658815    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.658895 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.658827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.658895 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.658863    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.660839 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.660812    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.661125 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.661089    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.666339 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.666323    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vx9b\" (UniqueName: \"kubernetes.io/projected/d8425fa5-385c-4ef1-ba0c-ce39cf30de55-kube-api-access-7vx9b\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ng874\" (UID: \"d8425fa5-385c-4ef1-ba0c-ce39cf30de55\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:36.744938 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:36.744909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:37.020687 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:37.020659    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"]
Sep 07 10:36:37.023124 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:36:37.023093    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8425fa5_385c_4ef1_ba0c_ce39cf30de55.slice/crio-828ac9a2ec0283d3f61f1f91f2f964fd014b0f208f988b74de633cb369bf13cb WatchSource:0}: Error finding container 828ac9a2ec0283d3f61f1f91f2f964fd014b0f208f988b74de633cb369bf13cb: Status 404 returned error can't find the container with id 828ac9a2ec0283d3f61f1f91f2f964fd014b0f208f988b74de633cb369bf13cb
Sep 07 10:36:37.530182 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:37.530152    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:36:37.639115 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:37.639073    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874" event={"ID":"d8425fa5-385c-4ef1-ba0c-ce39cf30de55","Type":"ContainerStarted","Data":"828ac9a2ec0283d3f61f1f91f2f964fd014b0f208f988b74de633cb369bf13cb"}
Sep 07 10:36:38.646338 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:38.646291    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874" event={"ID":"d8425fa5-385c-4ef1-ba0c-ce39cf30de55","Type":"ContainerStarted","Data":"ec7bb80560606c4817f6bd572f9374d31f1c9558fa28d2a4a119c09bda7f80c4"}
Sep 07 10:36:38.666765 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:38.666717    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874" podStartSLOduration=1.700168613 podStartE2EDuration="2.666701388s" totalImagesPullingTime="966.532775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:36:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:36:38.664192701 +0000 UTC m=+1002.398366933" watchObservedRunningTime="2026-09-07 10:36:38.666701388 +0000 UTC m=+1002.400875640"
Sep 07 10:36:41.754720 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:41.754690    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:36:43.034218 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:43.034183    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:36:46.746049 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:46.746013    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:46.746459 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:46.746087    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:46.758551 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:46.758518    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:47.707605 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:47.707569    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ng874"
Sep 07 10:36:51.239262 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.239226    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"]
Sep 07 10:36:51.243148 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.243112    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.245721 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.245699    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 07 10:36:51.251725 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.251702    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"]
Sep 07 10:36:51.261549 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.261517    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:36:51.388105 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.388069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426a17f8-51a8-49a8-8724-ea3f680dbd0c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.388301 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.388117    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426a17f8-51a8-49a8-8724-ea3f680dbd0c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.388301 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.388255    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426a17f8-51a8-49a8-8724-ea3f680dbd0c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.388301 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.388288    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426a17f8-51a8-49a8-8724-ea3f680dbd0c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.388405 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.388352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426a17f8-51a8-49a8-8724-ea3f680dbd0c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.388405 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.388373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7nb4\" (UniqueName: \"kubernetes.io/projected/426a17f8-51a8-49a8-8724-ea3f680dbd0c-kube-api-access-g7nb4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.489607 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.489513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426a17f8-51a8-49a8-8724-ea3f680dbd0c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.489607 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.489582    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426a17f8-51a8-49a8-8724-ea3f680dbd0c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.489825 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.489635    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426a17f8-51a8-49a8-8724-ea3f680dbd0c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.489825 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.489668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7nb4\" (UniqueName: \"kubernetes.io/projected/426a17f8-51a8-49a8-8724-ea3f680dbd0c-kube-api-access-g7nb4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.489825 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.489705    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426a17f8-51a8-49a8-8724-ea3f680dbd0c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.489825 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.489761    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426a17f8-51a8-49a8-8724-ea3f680dbd0c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.490038 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.489962    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426a17f8-51a8-49a8-8724-ea3f680dbd0c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.490038 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.489992    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426a17f8-51a8-49a8-8724-ea3f680dbd0c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.490162 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.490063    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426a17f8-51a8-49a8-8724-ea3f680dbd0c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.492200 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.492175    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426a17f8-51a8-49a8-8724-ea3f680dbd0c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.492586 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.492564    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426a17f8-51a8-49a8-8724-ea3f680dbd0c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.499703 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.499678    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7nb4\" (UniqueName: \"kubernetes.io/projected/426a17f8-51a8-49a8-8724-ea3f680dbd0c-kube-api-access-g7nb4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829\" (UID: \"426a17f8-51a8-49a8-8724-ea3f680dbd0c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.554720 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.554680    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:36:51.837391 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:51.837360    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"]
Sep 07 10:36:51.838484 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:36:51.838453    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod426a17f8_51a8_49a8_8724_ea3f680dbd0c.slice/crio-04891569a847c7894a72681f82f16c0214f9a508df7b0f30da330314fac667c9 WatchSource:0}: Error finding container 04891569a847c7894a72681f82f16c0214f9a508df7b0f30da330314fac667c9: Status 404 returned error can't find the container with id 04891569a847c7894a72681f82f16c0214f9a508df7b0f30da330314fac667c9
Sep 07 10:36:52.724415 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:52.724380    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829" event={"ID":"426a17f8-51a8-49a8-8724-ea3f680dbd0c","Type":"ContainerStarted","Data":"4445af9fa48159b695205a5cb8740273a31bc8d6c1ad0d48007bf1217d3f9ec2"}
Sep 07 10:36:52.724777 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:52.724439    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829" event={"ID":"426a17f8-51a8-49a8-8724-ea3f680dbd0c","Type":"ContainerStarted","Data":"04891569a847c7894a72681f82f16c0214f9a508df7b0f30da330314fac667c9"}
Sep 07 10:36:52.745873 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:52.745847    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:36:52.751794 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:36:52.751750    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829" podStartSLOduration=1.558343599 podStartE2EDuration="1.75173584s" totalImagesPullingTime="193.392241ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:36:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:36:52.750429527 +0000 UTC m=+1016.484603761" watchObservedRunningTime="2026-09-07 10:36:52.75173584 +0000 UTC m=+1016.485910074"
Sep 07 10:37:01.555306 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:37:01.555270    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:37:01.555788 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:37:01.555331    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:37:01.567708 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:37:01.567685    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:37:01.787978 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:37:01.787946    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829"
Sep 07 10:37:08.369062 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:37:08.369018    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:37:10.334297 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:37:10.334260    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:37:17.858712 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:37:17.858674    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:37:20.634609 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:37:20.634574    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:37:21.834931 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:37:21.834894    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:37:21.890210 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:37:21.890175    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:09.841715 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:09.841631    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:09.938756 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:09.938705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:10.994784 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:10.994749    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:11.085986 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:11.085951    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:28.784090 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:28.784053    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:28.974390 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:28.974357    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:29.170256 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:29.170183    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:29.333496 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:29.333459    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:30.450403 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:30.450365    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:30.522825 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:30.522779    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:30.927276 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:30.927200    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7762-140466978012976"]
Sep 07 10:38:30.930695 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:30.930678    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7762-140466978012976"
Sep 07 10:38:30.933117 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:30.933094    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-xwqv8\""
Sep 07 10:38:30.937196 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:30.937173    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7762-140466978012976"]
Sep 07 10:38:30.961290 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:30.961263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:31.041082 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:31.041052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcpgp\" (UniqueName: \"kubernetes.io/projected/81ffde71-adb5-4238-8329-210c9830218e-kube-api-access-rcpgp\") pod \"test-curl-7762-140466978012976\" (UID: \"81ffde71-adb5-4238-8329-210c9830218e\") " pod="opendatahub/test-curl-7762-140466978012976"
Sep 07 10:38:31.142198 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:31.142167    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcpgp\" (UniqueName: \"kubernetes.io/projected/81ffde71-adb5-4238-8329-210c9830218e-kube-api-access-rcpgp\") pod \"test-curl-7762-140466978012976\" (UID: \"81ffde71-adb5-4238-8329-210c9830218e\") " pod="opendatahub/test-curl-7762-140466978012976"
Sep 07 10:38:31.151420 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:31.151402    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcpgp\" (UniqueName: \"kubernetes.io/projected/81ffde71-adb5-4238-8329-210c9830218e-kube-api-access-rcpgp\") pod \"test-curl-7762-140466978012976\" (UID: \"81ffde71-adb5-4238-8329-210c9830218e\") " pod="opendatahub/test-curl-7762-140466978012976"
Sep 07 10:38:31.240430 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:31.240401    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7762-140466978012976"
Sep 07 10:38:31.515684 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:31.515659    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7762-140466978012976"]
Sep 07 10:38:31.517866 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:38:31.517826    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81ffde71_adb5_4238_8329_210c9830218e.slice/crio-cfba2d1d8ab3926eb74aecfc4d347f8ac94d79a64a371136bfa792325c460a18 WatchSource:0}: Error finding container cfba2d1d8ab3926eb74aecfc4d347f8ac94d79a64a371136bfa792325c460a18: Status 404 returned error can't find the container with id cfba2d1d8ab3926eb74aecfc4d347f8ac94d79a64a371136bfa792325c460a18
Sep 07 10:38:31.937225 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:31.937097    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:32.051239 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:32.051204    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:32.282856 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:32.282826    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7762-140466978012976" event={"ID":"81ffde71-adb5-4238-8329-210c9830218e","Type":"ContainerStarted","Data":"cfba2d1d8ab3926eb74aecfc4d347f8ac94d79a64a371136bfa792325c460a18"}
Sep 07 10:38:32.535549 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:32.535470    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:38:33.060188 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.060154    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"]
Sep 07 10:38:33.063399 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.063379    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:33.067762 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.067742    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-1ef687-serving-cert\""
Sep 07 10:38:33.078929 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.078894    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"]
Sep 07 10:38:33.098149 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.098092    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:33.156794 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.156768    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/829d7efb-68ac-4b7f-af51-59d9d763633d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:33.156888 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.156804    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7vhg\" (UniqueName: \"kubernetes.io/projected/829d7efb-68ac-4b7f-af51-59d9d763633d-kube-api-access-f7vhg\") pod \"maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:33.156888 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.156860    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829d7efb-68ac-4b7f-af51-59d9d763633d-maas-api-tls\") pod \"maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:33.156959 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.156902    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/829d7efb-68ac-4b7f-af51-59d9d763633d-metrics-certs\") pod \"maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:33.197481 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.197238    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:33.258012 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.257984    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/829d7efb-68ac-4b7f-af51-59d9d763633d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:33.258012 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.258015    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7vhg\" (UniqueName: \"kubernetes.io/projected/829d7efb-68ac-4b7f-af51-59d9d763633d-kube-api-access-f7vhg\") pod \"maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:33.258196 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.258040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829d7efb-68ac-4b7f-af51-59d9d763633d-maas-api-tls\") pod \"maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:33.258196 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.258061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/829d7efb-68ac-4b7f-af51-59d9d763633d-metrics-certs\") pod \"maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:33.258196 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:38:33.258173    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-serving-cert: secret "maas-api-e2e-inf-a-1ef687-serving-cert" not found
Sep 07 10:38:33.258313 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:38:33.258258    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/829d7efb-68ac-4b7f-af51-59d9d763633d-maas-api-tls podName:829d7efb-68ac-4b7f-af51-59d9d763633d nodeName:}" failed. No retries permitted until 2026-09-07 10:38:33.758238287 +0000 UTC m=+1117.492412502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/829d7efb-68ac-4b7f-af51-59d9d763633d-maas-api-tls") pod "maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4" (UID: "829d7efb-68ac-4b7f-af51-59d9d763633d") : secret "maas-api-e2e-inf-a-1ef687-serving-cert" not found
Sep 07 10:38:33.258596 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.258571    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/829d7efb-68ac-4b7f-af51-59d9d763633d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:33.260440 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.260424    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/829d7efb-68ac-4b7f-af51-59d9d763633d-metrics-certs\") pod \"maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:33.269014 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.268988    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7vhg\" (UniqueName: \"kubernetes.io/projected/829d7efb-68ac-4b7f-af51-59d9d763633d-kube-api-access-f7vhg\") pod \"maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:33.291370 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.291342    2577 generic.go:356] "Generic (PLEG): container finished" podID="81ffde71-adb5-4238-8329-210c9830218e" containerID="380c159c2c4e559e9e803e0b10a47263ea704ff9935ea8c2e012cc591bc379b6" exitCode=0
Sep 07 10:38:33.291485 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.291397    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7762-140466978012976" event={"ID":"81ffde71-adb5-4238-8329-210c9830218e","Type":"ContainerDied","Data":"380c159c2c4e559e9e803e0b10a47263ea704ff9935ea8c2e012cc591bc379b6"}
Sep 07 10:38:33.390457 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.390395    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:33.706652 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.706621    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:33.762763 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.762735    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829d7efb-68ac-4b7f-af51-59d9d763633d-maas-api-tls\") pod \"maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:33.765236 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.765209    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829d7efb-68ac-4b7f-af51-59d9d763633d-maas-api-tls\") pod \"maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:33.807922 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.807894    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:33.981635 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:33.981561    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:34.252503 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.252476    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"]
Sep 07 10:38:34.253426 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:38:34.253397    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod829d7efb_68ac_4b7f_af51_59d9d763633d.slice/crio-151c198fa4ab6565afa7a2dbac5916ba531b22e96945a3e686fadcf071448ada WatchSource:0}: Error finding container 151c198fa4ab6565afa7a2dbac5916ba531b22e96945a3e686fadcf071448ada: Status 404 returned error can't find the container with id 151c198fa4ab6565afa7a2dbac5916ba531b22e96945a3e686fadcf071448ada
Sep 07 10:38:34.298181 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.298143    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4" event={"ID":"829d7efb-68ac-4b7f-af51-59d9d763633d","Type":"ContainerStarted","Data":"151c198fa4ab6565afa7a2dbac5916ba531b22e96945a3e686fadcf071448ada"}
Sep 07 10:38:34.433532 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.433507    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7762-140466978012976"
Sep 07 10:38:34.453218 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.453196    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7762-140466978012976_81ffde71-adb5-4238-8329-210c9830218e/test-curl-7762-140466978012976/0.log"
Sep 07 10:38:34.484511 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.484482    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7762-140466978012976"]
Sep 07 10:38:34.491061 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.490997    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7762-140466978012976"]
Sep 07 10:38:34.570988 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.570929    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcpgp\" (UniqueName: \"kubernetes.io/projected/81ffde71-adb5-4238-8329-210c9830218e-kube-api-access-rcpgp\") pod \"81ffde71-adb5-4238-8329-210c9830218e\" (UID: \"81ffde71-adb5-4238-8329-210c9830218e\") "
Sep 07 10:38:34.573815 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.573779    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81ffde71-adb5-4238-8329-210c9830218e-kube-api-access-rcpgp" (OuterVolumeSpecName: "kube-api-access-rcpgp") pod "81ffde71-adb5-4238-8329-210c9830218e" (UID: "81ffde71-adb5-4238-8329-210c9830218e"). InnerVolumeSpecName "kube-api-access-rcpgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:38:34.615462 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.615427    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7762-140466978012976"]
Sep 07 10:38:34.615885 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.615866    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81ffde71-adb5-4238-8329-210c9830218e" containerName="test-curl-7762-140466978012976"
Sep 07 10:38:34.615968 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.615887    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="81ffde71-adb5-4238-8329-210c9830218e" containerName="test-curl-7762-140466978012976"
Sep 07 10:38:34.616023 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.615997    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="81ffde71-adb5-4238-8329-210c9830218e" containerName="test-curl-7762-140466978012976"
Sep 07 10:38:34.619123 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.619104    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7762-140466978012976"
Sep 07 10:38:34.626250 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.626049    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7762-140466978012976"]
Sep 07 10:38:34.671823 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.671793    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcpgp\" (UniqueName: \"kubernetes.io/projected/81ffde71-adb5-4238-8329-210c9830218e-kube-api-access-rcpgp\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:38:34.772540 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.772505    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5rqp\" (UniqueName: \"kubernetes.io/projected/932e16fc-665c-40a6-9340-51ac253c4a8a-kube-api-access-t5rqp\") pod \"test-curl-7762-140466978012976\" (UID: \"932e16fc-665c-40a6-9340-51ac253c4a8a\") " pod="opendatahub/test-curl-7762-140466978012976"
Sep 07 10:38:34.787100 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.787075    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81ffde71-adb5-4238-8329-210c9830218e" path="/var/lib/kubelet/pods/81ffde71-adb5-4238-8329-210c9830218e/volumes"
Sep 07 10:38:34.873833 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.873744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5rqp\" (UniqueName: \"kubernetes.io/projected/932e16fc-665c-40a6-9340-51ac253c4a8a-kube-api-access-t5rqp\") pod \"test-curl-7762-140466978012976\" (UID: \"932e16fc-665c-40a6-9340-51ac253c4a8a\") " pod="opendatahub/test-curl-7762-140466978012976"
Sep 07 10:38:34.882355 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.882328    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5rqp\" (UniqueName: \"kubernetes.io/projected/932e16fc-665c-40a6-9340-51ac253c4a8a-kube-api-access-t5rqp\") pod \"test-curl-7762-140466978012976\" (UID: \"932e16fc-665c-40a6-9340-51ac253c4a8a\") " pod="opendatahub/test-curl-7762-140466978012976"
Sep 07 10:38:34.929389 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:34.929363    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7762-140466978012976"
Sep 07 10:38:35.202395 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:35.202370    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7762-140466978012976"]
Sep 07 10:38:35.203858 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:38:35.203827    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod932e16fc_665c_40a6_9340_51ac253c4a8a.slice/crio-a70a8547b0a6f0918590e903d0ef293d4733304d35259a0327cf484f4c745ae1 WatchSource:0}: Error finding container a70a8547b0a6f0918590e903d0ef293d4733304d35259a0327cf484f4c745ae1: Status 404 returned error can't find the container with id a70a8547b0a6f0918590e903d0ef293d4733304d35259a0327cf484f4c745ae1
Sep 07 10:38:35.305437 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:35.305403    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4" event={"ID":"829d7efb-68ac-4b7f-af51-59d9d763633d","Type":"ContainerStarted","Data":"ef5ddb76220015be93f8860163027d2f79abae4b3ba5a1e2ec95fda08c709851"}
Sep 07 10:38:35.305608 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:35.305474    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:35.306769 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:35.306744    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7762-140466978012976" event={"ID":"932e16fc-665c-40a6-9340-51ac253c4a8a","Type":"ContainerStarted","Data":"a70a8547b0a6f0918590e903d0ef293d4733304d35259a0327cf484f4c745ae1"}
Sep 07 10:38:35.308027 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:35.308010    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7762-140466978012976"
Sep 07 10:38:35.308116 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:35.308071    2577 scope.go:117] "RemoveContainer" containerID="380c159c2c4e559e9e803e0b10a47263ea704ff9935ea8c2e012cc591bc379b6"
Sep 07 10:38:35.324326 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:35.324287    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4" podStartSLOduration=1.889162682 podStartE2EDuration="2.324272914s" totalImagesPullingTime="435.110232ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:38:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:38:35.322031495 +0000 UTC m=+1119.056205725" watchObservedRunningTime="2026-09-07 10:38:35.324272914 +0000 UTC m=+1119.058447141"
Sep 07 10:38:36.316328 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:36.316294    2577 generic.go:356] "Generic (PLEG): container finished" podID="932e16fc-665c-40a6-9340-51ac253c4a8a" containerID="e32875e39bbd42c402fc2ad57b544c653f50bd9eadf1485fdbf9b9c8508ed49b" exitCode=0
Sep 07 10:38:36.316761 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:36.316383    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7762-140466978012976" event={"ID":"932e16fc-665c-40a6-9340-51ac253c4a8a","Type":"ContainerDied","Data":"e32875e39bbd42c402fc2ad57b544c653f50bd9eadf1485fdbf9b9c8508ed49b"}
Sep 07 10:38:36.650857 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:36.650782    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:36.752326 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:36.752293    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:36.947281 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:36.947246    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:37.203814 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:37.203713    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:37.469880 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:37.469852    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7762-140466978012976"
Sep 07 10:38:37.491964 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:37.491936    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7762-140466978012976_932e16fc-665c-40a6-9340-51ac253c4a8a/test-curl-7762-140466978012976/0.log"
Sep 07 10:38:37.516904 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:37.516875    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7762-140466978012976"]
Sep 07 10:38:37.522465 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:37.522438    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7762-140466978012976"]
Sep 07 10:38:37.535793 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:37.535745    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:37.598784 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:37.598753    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5rqp\" (UniqueName: \"kubernetes.io/projected/932e16fc-665c-40a6-9340-51ac253c4a8a-kube-api-access-t5rqp\") pod \"932e16fc-665c-40a6-9340-51ac253c4a8a\" (UID: \"932e16fc-665c-40a6-9340-51ac253c4a8a\") "
Sep 07 10:38:37.600985 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:37.600956    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/932e16fc-665c-40a6-9340-51ac253c4a8a-kube-api-access-t5rqp" (OuterVolumeSpecName: "kube-api-access-t5rqp") pod "932e16fc-665c-40a6-9340-51ac253c4a8a" (UID: "932e16fc-665c-40a6-9340-51ac253c4a8a"). InnerVolumeSpecName "kube-api-access-t5rqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:38:37.700143 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:37.700090    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5rqp\" (UniqueName: \"kubernetes.io/projected/932e16fc-665c-40a6-9340-51ac253c4a8a-kube-api-access-t5rqp\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:38:38.051563 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:38.051528    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:38.331124 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:38.331042    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7762-140466978012976"
Sep 07 10:38:38.331124 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:38.331088    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a70a8547b0a6f0918590e903d0ef293d4733304d35259a0327cf484f4c745ae1"
Sep 07 10:38:38.342544 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:38:38.342512    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7762-140466978012976\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="932e16fc-665c-40a6-9340-51ac253c4a8a" pod="opendatahub/test-curl-7762-140466978012976"
Sep 07 10:38:38.392318 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:38.392286    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:38.668463 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:38.668379    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:38.786396 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:38.786365    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="932e16fc-665c-40a6-9340-51ac253c4a8a" path="/var/lib/kubelet/pods/932e16fc-665c-40a6-9340-51ac253c4a8a/volumes"
Sep 07 10:38:39.224370 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:39.224330    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:39.324934 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:39.324825    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:40.378701 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.378666    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"]
Sep 07 10:38:40.379318 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.379292    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="932e16fc-665c-40a6-9340-51ac253c4a8a" containerName="test-curl-7762-140466978012976"
Sep 07 10:38:40.379318 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.379317    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="932e16fc-665c-40a6-9340-51ac253c4a8a" containerName="test-curl-7762-140466978012976"
Sep 07 10:38:40.379538 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.379462    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="932e16fc-665c-40a6-9340-51ac253c4a8a" containerName="test-curl-7762-140466978012976"
Sep 07 10:38:40.382897 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.382874    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:40.385485 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.385460    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ea745819-serving-cert\""
Sep 07 10:38:40.395381 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.395347    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"]
Sep 07 10:38:40.414344 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.414313    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:40.513631 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.513590    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:40.527027 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.526996    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-maas-api-tls\") pod \"maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:40.527186 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.527040    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-openshift-service-ca\") pod \"maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:40.527248 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.527230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-metrics-certs\") pod \"maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:40.527287 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.527262    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c78tv\" (UniqueName: \"kubernetes.io/projected/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-kube-api-access-c78tv\") pod \"maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:40.628102 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.628057    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-metrics-certs\") pod \"maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:40.628102 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.628103    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c78tv\" (UniqueName: \"kubernetes.io/projected/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-kube-api-access-c78tv\") pod \"maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:40.628355 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.628148    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-maas-api-tls\") pod \"maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:40.628355 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.628208    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-openshift-service-ca\") pod \"maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:40.628355 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:38:40.628226    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-serving-cert: secret "maas-api-e2e-ait-ea745819-serving-cert" not found
Sep 07 10:38:40.628355 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:38:40.628275    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-maas-api-tls podName:23ffdb6a-37af-4403-95f9-b3891ae5b0c7 nodeName:}" failed. No retries permitted until 2026-09-07 10:38:41.128259007 +0000 UTC m=+1124.862433240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-maas-api-tls") pod "maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt" (UID: "23ffdb6a-37af-4403-95f9-b3891ae5b0c7") : secret "maas-api-e2e-ait-ea745819-serving-cert" not found
Sep 07 10:38:40.628903 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.628860    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-openshift-service-ca\") pod \"maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:40.630703 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.630685    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-metrics-certs\") pod \"maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:40.637312 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.637287    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c78tv\" (UniqueName: \"kubernetes.io/projected/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-kube-api-access-c78tv\") pod \"maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:40.922641 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:40.922563    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:41.021925 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:41.021893    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:41.111833 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:41.111794    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:41.132785 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:41.132750    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-maas-api-tls\") pod \"maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:41.135369 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:41.135344    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-maas-api-tls\") pod \"maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:41.294237 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:41.294201    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:41.322895 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:41.322873    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:38:41.569855 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:41.569832    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"]
Sep 07 10:38:41.572379 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:38:41.572350    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23ffdb6a_37af_4403_95f9_b3891ae5b0c7.slice/crio-82496283cfb8ffbdc8dadc1e99fed9c22d057a4d0f08017e27e546eaaad574fc WatchSource:0}: Error finding container 82496283cfb8ffbdc8dadc1e99fed9c22d057a4d0f08017e27e546eaaad574fc: Status 404 returned error can't find the container with id 82496283cfb8ffbdc8dadc1e99fed9c22d057a4d0f08017e27e546eaaad574fc
Sep 07 10:38:41.847764 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:41.847664    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:41.939918 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:41.939881    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:42.362334 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:42.362305    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt" event={"ID":"23ffdb6a-37af-4403-95f9-b3891ae5b0c7","Type":"ContainerStarted","Data":"ed0eb695372aead93a6135ce050a1715bf7163e77d26d38306b5b5e80a843723"}
Sep 07 10:38:42.362519 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:42.362356    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt" event={"ID":"23ffdb6a-37af-4403-95f9-b3891ae5b0c7","Type":"ContainerStarted","Data":"82496283cfb8ffbdc8dadc1e99fed9c22d057a4d0f08017e27e546eaaad574fc"}
Sep 07 10:38:42.362519 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:42.362417    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:42.380981 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:42.380936    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt" podStartSLOduration=2.048981717 podStartE2EDuration="2.380919651s" totalImagesPullingTime="331.937934ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:38:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:38:42.37934407 +0000 UTC m=+1126.113518306" watchObservedRunningTime="2026-09-07 10:38:42.380919651 +0000 UTC m=+1126.115093884"
Sep 07 10:38:43.008479 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.008448    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"]
Sep 07 10:38:43.012272 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.012244    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:43.015234 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.015208    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-c5b29a-serving-cert\""
Sep 07 10:38:43.032075 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.032053    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"]
Sep 07 10:38:43.047422 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.047398    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98d0f11c-f25d-4960-b19c-1634f6b4e202-maas-api-tls\") pod \"maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:43.047537 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.047452    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98d0f11c-f25d-4960-b19c-1634f6b4e202-openshift-service-ca\") pod \"maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:43.047537 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.047517    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98d0f11c-f25d-4960-b19c-1634f6b4e202-metrics-certs\") pod \"maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:43.047612 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.047555    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn78f\" (UniqueName: \"kubernetes.io/projected/98d0f11c-f25d-4960-b19c-1634f6b4e202-kube-api-access-vn78f\") pod \"maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:43.110298 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.110257    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:43.148071 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.148036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98d0f11c-f25d-4960-b19c-1634f6b4e202-metrics-certs\") pod \"maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:43.148071 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.148071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn78f\" (UniqueName: \"kubernetes.io/projected/98d0f11c-f25d-4960-b19c-1634f6b4e202-kube-api-access-vn78f\") pod \"maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:43.148412 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.148126    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98d0f11c-f25d-4960-b19c-1634f6b4e202-maas-api-tls\") pod \"maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:43.148412 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.148209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98d0f11c-f25d-4960-b19c-1634f6b4e202-openshift-service-ca\") pod \"maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:43.148824 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.148804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98d0f11c-f25d-4960-b19c-1634f6b4e202-openshift-service-ca\") pod \"maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:43.150592 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.150572    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98d0f11c-f25d-4960-b19c-1634f6b4e202-metrics-certs\") pod \"maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:43.150680 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.150647    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98d0f11c-f25d-4960-b19c-1634f6b4e202-maas-api-tls\") pod \"maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:43.166063 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.166037    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn78f\" (UniqueName: \"kubernetes.io/projected/98d0f11c-f25d-4960-b19c-1634f6b4e202-kube-api-access-vn78f\") pod \"maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:43.218492 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.218464    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:43.323114 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.323042    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:43.603986 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.603961    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"]
Sep 07 10:38:43.607238 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:38:43.607195    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98d0f11c_f25d_4960_b19c_1634f6b4e202.slice/crio-45920f9612c28534b9fa3218dafce339a78d62e9623330bf6da8630ee4dafdd1 WatchSource:0}: Error finding container 45920f9612c28534b9fa3218dafce339a78d62e9623330bf6da8630ee4dafdd1: Status 404 returned error can't find the container with id 45920f9612c28534b9fa3218dafce339a78d62e9623330bf6da8630ee4dafdd1
Sep 07 10:38:43.629528 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.629499    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:43.733413 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.733371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:43.755633 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.755594    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"]
Sep 07 10:38:43.760494 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.760477    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"
Sep 07 10:38:43.763323 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.763296    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-c5b29a\""
Sep 07 10:38:43.770964 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.770908    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"]
Sep 07 10:38:43.854831 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.854750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6pw6\" (UniqueName: \"kubernetes.io/projected/2a8f26c8-acde-4d38-b29d-4e32b043f712-kube-api-access-c6pw6\") pod \"payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8\" (UID: \"2a8f26c8-acde-4d38-b29d-4e32b043f712\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"
Sep 07 10:38:43.854976 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.854887    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a8f26c8-acde-4d38-b29d-4e32b043f712-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8\" (UID: \"2a8f26c8-acde-4d38-b29d-4e32b043f712\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"
Sep 07 10:38:43.956205 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.956177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6pw6\" (UniqueName: \"kubernetes.io/projected/2a8f26c8-acde-4d38-b29d-4e32b043f712-kube-api-access-c6pw6\") pod \"payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8\" (UID: \"2a8f26c8-acde-4d38-b29d-4e32b043f712\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"
Sep 07 10:38:43.956358 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.956243    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a8f26c8-acde-4d38-b29d-4e32b043f712-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8\" (UID: \"2a8f26c8-acde-4d38-b29d-4e32b043f712\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"
Sep 07 10:38:43.956887 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.956860    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a8f26c8-acde-4d38-b29d-4e32b043f712-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8\" (UID: \"2a8f26c8-acde-4d38-b29d-4e32b043f712\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"
Sep 07 10:38:43.965866 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:43.965846    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6pw6\" (UniqueName: \"kubernetes.io/projected/2a8f26c8-acde-4d38-b29d-4e32b043f712-kube-api-access-c6pw6\") pod \"payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8\" (UID: \"2a8f26c8-acde-4d38-b29d-4e32b043f712\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"
Sep 07 10:38:44.032557 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:44.032533    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:38:44.076530 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:44.076496    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"
Sep 07 10:38:44.355156 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:44.355097    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"]
Sep 07 10:38:44.356259 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:38:44.356235    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a8f26c8_acde_4d38_b29d_4e32b043f712.slice/crio-a229425180796a2985fcaf1263f4b96c38b00af02b25b44d1f39ec17d3240d8c WatchSource:0}: Error finding container a229425180796a2985fcaf1263f4b96c38b00af02b25b44d1f39ec17d3240d8c: Status 404 returned error can't find the container with id a229425180796a2985fcaf1263f4b96c38b00af02b25b44d1f39ec17d3240d8c
Sep 07 10:38:44.380053 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:44.379974    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6" event={"ID":"98d0f11c-f25d-4960-b19c-1634f6b4e202","Type":"ContainerStarted","Data":"3f5cc503e3697ac709031372aec8f8d96e819e80d0d8b0d113becf217d8da880"}
Sep 07 10:38:44.380053 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:44.380039    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6" event={"ID":"98d0f11c-f25d-4960-b19c-1634f6b4e202","Type":"ContainerStarted","Data":"45920f9612c28534b9fa3218dafce339a78d62e9623330bf6da8630ee4dafdd1"}
Sep 07 10:38:44.380255 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:44.380070    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:44.381412 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:44.381388    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8" event={"ID":"2a8f26c8-acde-4d38-b29d-4e32b043f712","Type":"ContainerStarted","Data":"a229425180796a2985fcaf1263f4b96c38b00af02b25b44d1f39ec17d3240d8c"}
Sep 07 10:38:44.398252 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:44.398198    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6" podStartSLOduration=2.042387361 podStartE2EDuration="2.398182982s" totalImagesPullingTime="355.795621ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:38:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:38:44.396409585 +0000 UTC m=+1128.130583818" watchObservedRunningTime="2026-09-07 10:38:44.398182982 +0000 UTC m=+1128.132357214"
Sep 07 10:38:46.400434 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:46.400399    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8" event={"ID":"2a8f26c8-acde-4d38-b29d-4e32b043f712","Type":"ContainerStarted","Data":"13f97aa69149a1ea1fcbaf4f77310eb16035e88efbab0e05b7db73726085c813"}
Sep 07 10:38:46.400813 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:46.400498    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"
Sep 07 10:38:46.417585 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:46.417537    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8" podStartSLOduration=2.060934718 podStartE2EDuration="3.417497166s" totalImagesPullingTime="1.356562448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:38:46.415899178 +0000 UTC m=+1130.150073451" watchObservedRunningTime="2026-09-07 10:38:46.417497166 +0000 UTC m=+1130.151671399"
Sep 07 10:38:47.717765 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:47.717710    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:47.821881 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:47.821843    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:47.922849 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:47.922813    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:48.022652 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:48.022566    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:48.155480 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:48.155447    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:48.296257 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:48.296143    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:48.354445 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:48.354410    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:48.374938 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:48.374917    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:38:50.396417 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:50.396391    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:38:51.012809 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:51.012769    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:53.184045 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:53.184004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:54.047742 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:54.047700    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:54.153010 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:54.152969    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:54.249268 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:54.249235    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:54.441597 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:54.441491    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:54.567362 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:54.567326    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:56.865102 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:56.865063    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:57.290289 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:57.290243    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:57.409754 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:57.409726    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"
Sep 07 10:38:57.673641 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:57.673539    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:38:57.857843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:38:57.857807    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:00.140702 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:00.140668    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:39:00.465271 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:00.465223    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:00.565553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:00.565512    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:01.428787 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:01.428756    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:03.558453 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:03.558420    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:03.638454 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:03.638423    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:04.774302 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.774264    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"]
Sep 07 10:39:04.780632 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.780604    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:04.783268 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.783245    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-b4061f-serving-cert\""
Sep 07 10:39:04.790706 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.790681    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"]
Sep 07 10:39:04.809490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.809451    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:04.848489 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.848458    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8515b0d-0895-480e-825f-c1ddde9866d1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b4061f-849b888c44-z77j7\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:04.848678 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.848517    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8515b0d-0895-480e-825f-c1ddde9866d1-maas-api-tls\") pod \"maas-api-e2e-inf-b-b4061f-849b888c44-z77j7\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:04.848678 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.848583    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85gq4\" (UniqueName: \"kubernetes.io/projected/c8515b0d-0895-480e-825f-c1ddde9866d1-kube-api-access-85gq4\") pod \"maas-api-e2e-inf-b-b4061f-849b888c44-z77j7\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:04.848678 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.848668    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8515b0d-0895-480e-825f-c1ddde9866d1-metrics-certs\") pod \"maas-api-e2e-inf-b-b4061f-849b888c44-z77j7\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:04.870769 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.870733    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:04.950051 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.950022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85gq4\" (UniqueName: \"kubernetes.io/projected/c8515b0d-0895-480e-825f-c1ddde9866d1-kube-api-access-85gq4\") pod \"maas-api-e2e-inf-b-b4061f-849b888c44-z77j7\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:04.950255 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.950105    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8515b0d-0895-480e-825f-c1ddde9866d1-metrics-certs\") pod \"maas-api-e2e-inf-b-b4061f-849b888c44-z77j7\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:04.950255 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.950173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8515b0d-0895-480e-825f-c1ddde9866d1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b4061f-849b888c44-z77j7\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:04.950255 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.950200    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8515b0d-0895-480e-825f-c1ddde9866d1-maas-api-tls\") pod \"maas-api-e2e-inf-b-b4061f-849b888c44-z77j7\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:04.950852 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.950816    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8515b0d-0895-480e-825f-c1ddde9866d1-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b4061f-849b888c44-z77j7\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:04.952798 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.952767    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8515b0d-0895-480e-825f-c1ddde9866d1-maas-api-tls\") pod \"maas-api-e2e-inf-b-b4061f-849b888c44-z77j7\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:04.952897 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.952773    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8515b0d-0895-480e-825f-c1ddde9866d1-metrics-certs\") pod \"maas-api-e2e-inf-b-b4061f-849b888c44-z77j7\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:04.960812 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.960792    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85gq4\" (UniqueName: \"kubernetes.io/projected/c8515b0d-0895-480e-825f-c1ddde9866d1-kube-api-access-85gq4\") pod \"maas-api-e2e-inf-b-b4061f-849b888c44-z77j7\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:04.971934 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:04.971907    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:05.093453 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:05.093375    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:05.374948 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:05.374920    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"]
Sep 07 10:39:05.376416 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:39:05.376389    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8515b0d_0895_480e_825f_c1ddde9866d1.slice/crio-8ff4afdd94caf0c2e52b0dc43276776ae65aa61ec7f9eadd8db6860e105ff251 WatchSource:0}: Error finding container 8ff4afdd94caf0c2e52b0dc43276776ae65aa61ec7f9eadd8db6860e105ff251: Status 404 returned error can't find the container with id 8ff4afdd94caf0c2e52b0dc43276776ae65aa61ec7f9eadd8db6860e105ff251
Sep 07 10:39:05.434566 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:05.434533    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:05.517953 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:05.517923    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7" event={"ID":"c8515b0d-0895-480e-825f-c1ddde9866d1","Type":"ContainerStarted","Data":"8ff4afdd94caf0c2e52b0dc43276776ae65aa61ec7f9eadd8db6860e105ff251"}
Sep 07 10:39:05.529291 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:05.529259    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:05.908593 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:05.908546    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:06.524973 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:06.524943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7" event={"ID":"c8515b0d-0895-480e-825f-c1ddde9866d1","Type":"ContainerStarted","Data":"c7fa61887d2800dffca3299440be2ce3cd86bd2d028303be3863c1c6e46a8380"}
Sep 07 10:39:06.525211 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:06.525041    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:06.544410 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:06.544370    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7" podStartSLOduration=2.207168553 podStartE2EDuration="2.544354173s" totalImagesPullingTime="337.18562ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:39:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:39:06.5435212 +0000 UTC m=+1150.277695431" watchObservedRunningTime="2026-09-07 10:39:06.544354173 +0000 UTC m=+1150.278528406"
Sep 07 10:39:06.902578 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:06.902488    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:07.716981 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:07.716948    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:08.612615 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:08.612551    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:12.034843 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:12.034805    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:12.536125 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:12.536094    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:39:16.265836 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:16.265765    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:16.318995 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:16.318961    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:16.834570 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:16.834533    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:39:17.354792 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:17.354760    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:20.590861 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:20.590820    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:21.178913 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.178876    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"]
Sep 07 10:39:21.179410 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.179376    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt" podUID="23ffdb6a-37af-4403-95f9-b3891ae5b0c7" containerName="maas-api" containerID="cri-o://ed0eb695372aead93a6135ce050a1715bf7163e77d26d38306b5b5e80a843723" gracePeriod=30
Sep 07 10:39:21.203232 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.203122    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:21.288615 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.288569    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:21.387832 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.387788    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:21.439262 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.439202    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:39:21.462244 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.461755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:21.564453 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.564421    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:21.600733 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.600703    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c78tv\" (UniqueName: \"kubernetes.io/projected/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-kube-api-access-c78tv\") pod \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") "
Sep 07 10:39:21.601274 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.600753    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-metrics-certs\") pod \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") "
Sep 07 10:39:21.601274 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.600782    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-maas-api-tls\") pod \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") "
Sep 07 10:39:21.601274 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.600863    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-openshift-service-ca\") pod \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\" (UID: \"23ffdb6a-37af-4403-95f9-b3891ae5b0c7\") "
Sep 07 10:39:21.601444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.601287    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "23ffdb6a-37af-4403-95f9-b3891ae5b0c7" (UID: "23ffdb6a-37af-4403-95f9-b3891ae5b0c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:39:21.601658 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.601640    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:39:21.603401 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.603373    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "23ffdb6a-37af-4403-95f9-b3891ae5b0c7" (UID: "23ffdb6a-37af-4403-95f9-b3891ae5b0c7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:39:21.603696 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.603673    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-kube-api-access-c78tv" (OuterVolumeSpecName: "kube-api-access-c78tv") pod "23ffdb6a-37af-4403-95f9-b3891ae5b0c7" (UID: "23ffdb6a-37af-4403-95f9-b3891ae5b0c7"). InnerVolumeSpecName "kube-api-access-c78tv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:39:21.603696 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.603673    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "23ffdb6a-37af-4403-95f9-b3891ae5b0c7" (UID: "23ffdb6a-37af-4403-95f9-b3891ae5b0c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:39:21.618215 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.618189    2577 generic.go:356] "Generic (PLEG): container finished" podID="23ffdb6a-37af-4403-95f9-b3891ae5b0c7" containerID="ed0eb695372aead93a6135ce050a1715bf7163e77d26d38306b5b5e80a843723" exitCode=0
Sep 07 10:39:21.618353 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.618247    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"
Sep 07 10:39:21.618353 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.618258    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt" event={"ID":"23ffdb6a-37af-4403-95f9-b3891ae5b0c7","Type":"ContainerDied","Data":"ed0eb695372aead93a6135ce050a1715bf7163e77d26d38306b5b5e80a843723"}
Sep 07 10:39:21.618353 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.618311    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt" event={"ID":"23ffdb6a-37af-4403-95f9-b3891ae5b0c7","Type":"ContainerDied","Data":"82496283cfb8ffbdc8dadc1e99fed9c22d057a4d0f08017e27e546eaaad574fc"}
Sep 07 10:39:21.618353 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.618335    2577 scope.go:117] "RemoveContainer" containerID="ed0eb695372aead93a6135ce050a1715bf7163e77d26d38306b5b5e80a843723"
Sep 07 10:39:21.628873 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.628854    2577 scope.go:117] "RemoveContainer" containerID="ed0eb695372aead93a6135ce050a1715bf7163e77d26d38306b5b5e80a843723"
Sep 07 10:39:21.629200 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:39:21.629179    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed0eb695372aead93a6135ce050a1715bf7163e77d26d38306b5b5e80a843723\": container with ID starting with ed0eb695372aead93a6135ce050a1715bf7163e77d26d38306b5b5e80a843723 not found: ID does not exist" containerID="ed0eb695372aead93a6135ce050a1715bf7163e77d26d38306b5b5e80a843723"
Sep 07 10:39:21.629273 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.629209    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed0eb695372aead93a6135ce050a1715bf7163e77d26d38306b5b5e80a843723"} err="failed to get container status \"ed0eb695372aead93a6135ce050a1715bf7163e77d26d38306b5b5e80a843723\": rpc error: code = NotFound desc = could not find container \"ed0eb695372aead93a6135ce050a1715bf7163e77d26d38306b5b5e80a843723\": container with ID starting with ed0eb695372aead93a6135ce050a1715bf7163e77d26d38306b5b5e80a843723 not found: ID does not exist"
Sep 07 10:39:21.639850 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.639825    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"]
Sep 07 10:39:21.647745 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.647726    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ea745819-85648c5bcc-xxwnt"]
Sep 07 10:39:21.703045 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.703021    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:39:21.703045 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.703045    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c78tv\" (UniqueName: \"kubernetes.io/projected/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-kube-api-access-c78tv\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:39:21.703259 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:21.703054    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23ffdb6a-37af-4403-95f9-b3891ae5b0c7-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:39:22.054702 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:22.054609    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:22.148524 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:22.148471    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:22.277288 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:22.277251    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:22.635381 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:22.635333    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:22.785957 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:22.785927    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23ffdb6a-37af-4403-95f9-b3891ae5b0c7" path="/var/lib/kubelet/pods/23ffdb6a-37af-4403-95f9-b3891ae5b0c7/volumes"
Sep 07 10:39:22.826160 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:22.826104    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:25.758280 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:25.758239    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:25.860440 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:25.860403    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:25.957060 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:25.957020    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:26.669390 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.669362    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"]
Sep 07 10:39:26.669792 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.669779    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23ffdb6a-37af-4403-95f9-b3891ae5b0c7" containerName="maas-api"
Sep 07 10:39:26.669869 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.669793    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="23ffdb6a-37af-4403-95f9-b3891ae5b0c7" containerName="maas-api"
Sep 07 10:39:26.669869 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.669852    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="23ffdb6a-37af-4403-95f9-b3891ae5b0c7" containerName="maas-api"
Sep 07 10:39:26.673504 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.673481    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:26.676396 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.676337    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-6cd248c6-serving-cert\""
Sep 07 10:39:26.682175 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.681778    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"]
Sep 07 10:39:26.706801 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.706773    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:26.802198 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.802152    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:26.850460 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.850429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25429572-14f0-4af6-9980-3dd613de3e6d-maas-api-tls\") pod \"maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:26.850460 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.850462    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blphn\" (UniqueName: \"kubernetes.io/projected/25429572-14f0-4af6-9980-3dd613de3e6d-kube-api-access-blphn\") pod \"maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:26.850656 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.850575    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25429572-14f0-4af6-9980-3dd613de3e6d-openshift-service-ca\") pod \"maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:26.850656 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.850628    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25429572-14f0-4af6-9980-3dd613de3e6d-metrics-certs\") pod \"maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:26.951814 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.951785    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25429572-14f0-4af6-9980-3dd613de3e6d-metrics-certs\") pod \"maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:26.951984 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.951842    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25429572-14f0-4af6-9980-3dd613de3e6d-maas-api-tls\") pod \"maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:26.951984 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:39:26.951922    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-serving-cert: secret "maas-api-e2e-migrate-6cd248c6-serving-cert" not found
Sep 07 10:39:26.951984 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.951951    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blphn\" (UniqueName: \"kubernetes.io/projected/25429572-14f0-4af6-9980-3dd613de3e6d-kube-api-access-blphn\") pod \"maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:26.951984 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:39:26.951986    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25429572-14f0-4af6-9980-3dd613de3e6d-maas-api-tls podName:25429572-14f0-4af6-9980-3dd613de3e6d nodeName:}" failed. No retries permitted until 2026-09-07 10:39:27.451970109 +0000 UTC m=+1171.186144319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/25429572-14f0-4af6-9980-3dd613de3e6d-maas-api-tls") pod "maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd" (UID: "25429572-14f0-4af6-9980-3dd613de3e6d") : secret "maas-api-e2e-migrate-6cd248c6-serving-cert" not found
Sep 07 10:39:26.952269 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.952094    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25429572-14f0-4af6-9980-3dd613de3e6d-openshift-service-ca\") pod \"maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:26.952652 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.952633    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25429572-14f0-4af6-9980-3dd613de3e6d-openshift-service-ca\") pod \"maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:26.954436 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.954413    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25429572-14f0-4af6-9980-3dd613de3e6d-metrics-certs\") pod \"maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:26.964171 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:26.964144    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blphn\" (UniqueName: \"kubernetes.io/projected/25429572-14f0-4af6-9980-3dd613de3e6d-kube-api-access-blphn\") pod \"maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:27.186034 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:27.185990    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:27.282912 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:27.282831    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:27.444051 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:27.444011    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:27.457145 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:27.457094    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25429572-14f0-4af6-9980-3dd613de3e6d-maas-api-tls\") pod \"maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:27.459627 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:27.459608    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25429572-14f0-4af6-9980-3dd613de3e6d-maas-api-tls\") pod \"maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:27.588383 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:27.588290    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:27.688368 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:27.688332    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:27.867374 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:27.866925    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"]
Sep 07 10:39:28.455840 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:28.455803    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:28.610863 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:28.610831    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:28.670155 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:28.670098    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd" event={"ID":"25429572-14f0-4af6-9980-3dd613de3e6d","Type":"ContainerStarted","Data":"ff9a2340cec2c6c13c6c36ac0a597f6c1caa3dbb98d59dc1970742d202526da5"}
Sep 07 10:39:28.670320 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:28.670181    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd" event={"ID":"25429572-14f0-4af6-9980-3dd613de3e6d","Type":"ContainerStarted","Data":"0dc10d8be0b8824b318dd9ea7e2e139fd3e96dd7827b07b00204c350c89c4614"}
Sep 07 10:39:28.670402 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:28.670334    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:28.691780 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:28.691722    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd" podStartSLOduration=2.073591515 podStartE2EDuration="2.691705764s" totalImagesPullingTime="618.114249ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:39:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:39:28.688839306 +0000 UTC m=+1172.423013534" watchObservedRunningTime="2026-09-07 10:39:28.691705764 +0000 UTC m=+1172.425879996"
Sep 07 10:39:32.046521 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:32.046486    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:39:34.683593 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:34.683561    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:39:46.629627 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:46.629590    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:39:47.158491 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:47.158454    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:39:56.806364 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:56.806337    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:39:56.813119 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:39:56.813097    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:40:05.243970 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:05.243936    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:40:06.876748 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:06.876657    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"]
Sep 07 10:40:06.882218 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:06.882180    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd" podUID="25429572-14f0-4af6-9980-3dd613de3e6d" containerName="maas-api" containerID="cri-o://ff9a2340cec2c6c13c6c36ac0a597f6c1caa3dbb98d59dc1970742d202526da5" gracePeriod=30
Sep 07 10:40:06.896858 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:06.896826    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:06.998419 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:06.998350    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:07.097469 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.097422    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:07.143650 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.143626    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:40:07.167876 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.167835    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:07.220650 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.220617    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blphn\" (UniqueName: \"kubernetes.io/projected/25429572-14f0-4af6-9980-3dd613de3e6d-kube-api-access-blphn\") pod \"25429572-14f0-4af6-9980-3dd613de3e6d\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") "
Sep 07 10:40:07.220809 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.220698    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25429572-14f0-4af6-9980-3dd613de3e6d-maas-api-tls\") pod \"25429572-14f0-4af6-9980-3dd613de3e6d\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") "
Sep 07 10:40:07.220809 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.220732    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25429572-14f0-4af6-9980-3dd613de3e6d-metrics-certs\") pod \"25429572-14f0-4af6-9980-3dd613de3e6d\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") "
Sep 07 10:40:07.220809 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.220782    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25429572-14f0-4af6-9980-3dd613de3e6d-openshift-service-ca\") pod \"25429572-14f0-4af6-9980-3dd613de3e6d\" (UID: \"25429572-14f0-4af6-9980-3dd613de3e6d\") "
Sep 07 10:40:07.221325 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.221279    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25429572-14f0-4af6-9980-3dd613de3e6d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "25429572-14f0-4af6-9980-3dd613de3e6d" (UID: "25429572-14f0-4af6-9980-3dd613de3e6d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:40:07.222931 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.222897    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25429572-14f0-4af6-9980-3dd613de3e6d-kube-api-access-blphn" (OuterVolumeSpecName: "kube-api-access-blphn") pod "25429572-14f0-4af6-9980-3dd613de3e6d" (UID: "25429572-14f0-4af6-9980-3dd613de3e6d"). InnerVolumeSpecName "kube-api-access-blphn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:40:07.223027 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.222997    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25429572-14f0-4af6-9980-3dd613de3e6d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "25429572-14f0-4af6-9980-3dd613de3e6d" (UID: "25429572-14f0-4af6-9980-3dd613de3e6d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:40:07.223027 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.223009    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25429572-14f0-4af6-9980-3dd613de3e6d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "25429572-14f0-4af6-9980-3dd613de3e6d" (UID: "25429572-14f0-4af6-9980-3dd613de3e6d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:40:07.229688 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.229660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:07.322228 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.322199    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blphn\" (UniqueName: \"kubernetes.io/projected/25429572-14f0-4af6-9980-3dd613de3e6d-kube-api-access-blphn\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:40:07.322228 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.322228    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25429572-14f0-4af6-9980-3dd613de3e6d-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:40:07.322455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.322255    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25429572-14f0-4af6-9980-3dd613de3e6d-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:40:07.322455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.322267    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25429572-14f0-4af6-9980-3dd613de3e6d-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:40:07.382631 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.382519    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:07.917870 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.917836    2577 generic.go:356] "Generic (PLEG): container finished" podID="25429572-14f0-4af6-9980-3dd613de3e6d" containerID="ff9a2340cec2c6c13c6c36ac0a597f6c1caa3dbb98d59dc1970742d202526da5" exitCode=0
Sep 07 10:40:07.918323 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.917882    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"
Sep 07 10:40:07.918323 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.917929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd" event={"ID":"25429572-14f0-4af6-9980-3dd613de3e6d","Type":"ContainerDied","Data":"ff9a2340cec2c6c13c6c36ac0a597f6c1caa3dbb98d59dc1970742d202526da5"}
Sep 07 10:40:07.918323 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.917978    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd" event={"ID":"25429572-14f0-4af6-9980-3dd613de3e6d","Type":"ContainerDied","Data":"0dc10d8be0b8824b318dd9ea7e2e139fd3e96dd7827b07b00204c350c89c4614"}
Sep 07 10:40:07.918323 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.918003    2577 scope.go:117] "RemoveContainer" containerID="ff9a2340cec2c6c13c6c36ac0a597f6c1caa3dbb98d59dc1970742d202526da5"
Sep 07 10:40:07.928370 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.928343    2577 scope.go:117] "RemoveContainer" containerID="ff9a2340cec2c6c13c6c36ac0a597f6c1caa3dbb98d59dc1970742d202526da5"
Sep 07 10:40:07.928649 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:40:07.928627    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff9a2340cec2c6c13c6c36ac0a597f6c1caa3dbb98d59dc1970742d202526da5\": container with ID starting with ff9a2340cec2c6c13c6c36ac0a597f6c1caa3dbb98d59dc1970742d202526da5 not found: ID does not exist" containerID="ff9a2340cec2c6c13c6c36ac0a597f6c1caa3dbb98d59dc1970742d202526da5"
Sep 07 10:40:07.928711 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.928661    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff9a2340cec2c6c13c6c36ac0a597f6c1caa3dbb98d59dc1970742d202526da5"} err="failed to get container status \"ff9a2340cec2c6c13c6c36ac0a597f6c1caa3dbb98d59dc1970742d202526da5\": rpc error: code = NotFound desc = could not find container \"ff9a2340cec2c6c13c6c36ac0a597f6c1caa3dbb98d59dc1970742d202526da5\": container with ID starting with ff9a2340cec2c6c13c6c36ac0a597f6c1caa3dbb98d59dc1970742d202526da5 not found: ID does not exist"
Sep 07 10:40:07.939076 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.939053    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"]
Sep 07 10:40:07.942449 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:07.942431    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6cd248c6-89865b9c6-x9ldd"]
Sep 07 10:40:08.042474 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:08.042446    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:08.533277 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:08.533243    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:08.786360 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:08.786282    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25429572-14f0-4af6-9980-3dd613de3e6d" path="/var/lib/kubelet/pods/25429572-14f0-4af6-9980-3dd613de3e6d/volumes"
Sep 07 10:40:09.379167 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:09.379112    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:10.888971 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:10.888932    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:10.987777 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:10.987745    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:12.200264 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.200221    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"]
Sep 07 10:40:12.200932 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.200909    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25429572-14f0-4af6-9980-3dd613de3e6d" containerName="maas-api"
Sep 07 10:40:12.201079 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.201035    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="25429572-14f0-4af6-9980-3dd613de3e6d" containerName="maas-api"
Sep 07 10:40:12.201260 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.201235    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="25429572-14f0-4af6-9980-3dd613de3e6d" containerName="maas-api"
Sep 07 10:40:12.205992 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.205973    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:12.208307 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.208290    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fc9185fe-serving-cert\""
Sep 07 10:40:12.216598 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.216574    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"]
Sep 07 10:40:12.229263 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.229232    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:12.328668 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.328632    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:12.367207 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.367177    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11276480-897e-47df-b89d-931ec05f2afe-maas-api-tls\") pod \"maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:12.367373 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.367230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11276480-897e-47df-b89d-931ec05f2afe-metrics-certs\") pod \"maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:12.367373 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.367271    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11276480-897e-47df-b89d-931ec05f2afe-openshift-service-ca\") pod \"maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:12.367373 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.367286    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-567ph\" (UniqueName: \"kubernetes.io/projected/11276480-897e-47df-b89d-931ec05f2afe-kube-api-access-567ph\") pod \"maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:12.468717 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.468632    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11276480-897e-47df-b89d-931ec05f2afe-maas-api-tls\") pod \"maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:12.468717 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.468681    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11276480-897e-47df-b89d-931ec05f2afe-metrics-certs\") pod \"maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:12.468903 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:40:12.468793    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-serving-cert: secret "maas-api-e2e-ait-fc9185fe-serving-cert" not found
Sep 07 10:40:12.468903 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.468801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11276480-897e-47df-b89d-931ec05f2afe-openshift-service-ca\") pod \"maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:12.468903 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:40:12.468865    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11276480-897e-47df-b89d-931ec05f2afe-maas-api-tls podName:11276480-897e-47df-b89d-931ec05f2afe nodeName:}" failed. No retries permitted until 2026-09-07 10:40:12.968841539 +0000 UTC m=+1216.703015749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/11276480-897e-47df-b89d-931ec05f2afe-maas-api-tls") pod "maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf" (UID: "11276480-897e-47df-b89d-931ec05f2afe") : secret "maas-api-e2e-ait-fc9185fe-serving-cert" not found
Sep 07 10:40:12.469006 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.468908    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-567ph\" (UniqueName: \"kubernetes.io/projected/11276480-897e-47df-b89d-931ec05f2afe-kube-api-access-567ph\") pod \"maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:12.469545 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.469520    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11276480-897e-47df-b89d-931ec05f2afe-openshift-service-ca\") pod \"maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:12.471265 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.471247    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11276480-897e-47df-b89d-931ec05f2afe-metrics-certs\") pod \"maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:12.478392 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.478364    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-567ph\" (UniqueName: \"kubernetes.io/projected/11276480-897e-47df-b89d-931ec05f2afe-kube-api-access-567ph\") pod \"maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:12.728023 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.727930    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:12.822979 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.822896    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:12.906717 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.906686    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:12.974029 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.973994    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11276480-897e-47df-b89d-931ec05f2afe-maas-api-tls\") pod \"maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:12.976960 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:12.976923    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11276480-897e-47df-b89d-931ec05f2afe-maas-api-tls\") pod \"maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:13.119406 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:13.119317    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:13.421789 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:40:13.421758    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11276480_897e_47df_b89d_931ec05f2afe.slice/crio-2a0a51c55a0719db8203856a912cb49ce6e96de617545f26dfea705823673672 WatchSource:0}: Error finding container 2a0a51c55a0719db8203856a912cb49ce6e96de617545f26dfea705823673672: Status 404 returned error can't find the container with id 2a0a51c55a0719db8203856a912cb49ce6e96de617545f26dfea705823673672
Sep 07 10:40:13.422673 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:13.422650    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"]
Sep 07 10:40:13.956473 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:13.956437    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf" event={"ID":"11276480-897e-47df-b89d-931ec05f2afe","Type":"ContainerStarted","Data":"310d09782ff31b9f2de85de5d942279f02488f8f44b77b87421185242907b7be"}
Sep 07 10:40:13.956625 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:13.956491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf" event={"ID":"11276480-897e-47df-b89d-931ec05f2afe","Type":"ContainerStarted","Data":"2a0a51c55a0719db8203856a912cb49ce6e96de617545f26dfea705823673672"}
Sep 07 10:40:13.956625 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:13.956518    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:13.976708 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:13.976628    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf" podStartSLOduration=1.548102442 podStartE2EDuration="1.976603622s" totalImagesPullingTime="428.50118ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:40:13.975551392 +0000 UTC m=+1217.709725624" watchObservedRunningTime="2026-09-07 10:40:13.976603622 +0000 UTC m=+1217.710777857"
Sep 07 10:40:19.743887 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:19.743851    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:19.967902 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:19.967874    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:25.137779 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:25.137740    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:40:30.197310 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.197272    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"]
Sep 07 10:40:30.197792 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.197495    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6" podUID="98d0f11c-f25d-4960-b19c-1634f6b4e202" containerName="maas-api" containerID="cri-o://3f5cc503e3697ac709031372aec8f8d96e819e80d0d8b0d113becf217d8da880" gracePeriod=30
Sep 07 10:40:30.228237 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.228194    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:30.276928 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.276896    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"]
Sep 07 10:40:30.277624 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.277431    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8" podUID="2a8f26c8-acde-4d38-b29d-4e32b043f712" containerName="payload-pre-processing" containerID="cri-o://13f97aa69149a1ea1fcbaf4f77310eb16035e88efbab0e05b7db73726085c813" gracePeriod=30
Sep 07 10:40:30.326965 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.326915    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:30.392732 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.392696    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6" podUID="98d0f11c-f25d-4960-b19c-1634f6b4e202" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.63:8443/health\": dial tcp 10.132.0.63:8443: connect: connection refused"
Sep 07 10:40:30.428048 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.427999    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:30.483363 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.483266    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:30.542059 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.542028    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:40:30.574933 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.574906    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"
Sep 07 10:40:30.639860 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.639832    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98d0f11c-f25d-4960-b19c-1634f6b4e202-openshift-service-ca\") pod \"98d0f11c-f25d-4960-b19c-1634f6b4e202\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") "
Sep 07 10:40:30.640045 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.639914    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98d0f11c-f25d-4960-b19c-1634f6b4e202-maas-api-tls\") pod \"98d0f11c-f25d-4960-b19c-1634f6b4e202\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") "
Sep 07 10:40:30.640045 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.639937    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vn78f\" (UniqueName: \"kubernetes.io/projected/98d0f11c-f25d-4960-b19c-1634f6b4e202-kube-api-access-vn78f\") pod \"98d0f11c-f25d-4960-b19c-1634f6b4e202\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") "
Sep 07 10:40:30.640045 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.639990    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98d0f11c-f25d-4960-b19c-1634f6b4e202-metrics-certs\") pod \"98d0f11c-f25d-4960-b19c-1634f6b4e202\" (UID: \"98d0f11c-f25d-4960-b19c-1634f6b4e202\") "
Sep 07 10:40:30.640045 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.640013    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6pw6\" (UniqueName: \"kubernetes.io/projected/2a8f26c8-acde-4d38-b29d-4e32b043f712-kube-api-access-c6pw6\") pod \"2a8f26c8-acde-4d38-b29d-4e32b043f712\" (UID: \"2a8f26c8-acde-4d38-b29d-4e32b043f712\") "
Sep 07 10:40:30.640045 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.640043    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a8f26c8-acde-4d38-b29d-4e32b043f712-plugins-config-volume\") pod \"2a8f26c8-acde-4d38-b29d-4e32b043f712\" (UID: \"2a8f26c8-acde-4d38-b29d-4e32b043f712\") "
Sep 07 10:40:30.640370 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.640223    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98d0f11c-f25d-4960-b19c-1634f6b4e202-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98d0f11c-f25d-4960-b19c-1634f6b4e202" (UID: "98d0f11c-f25d-4960-b19c-1634f6b4e202"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:40:30.640424 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.640375    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98d0f11c-f25d-4960-b19c-1634f6b4e202-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:40:30.640580 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.640554    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a8f26c8-acde-4d38-b29d-4e32b043f712-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2a8f26c8-acde-4d38-b29d-4e32b043f712" (UID: "2a8f26c8-acde-4d38-b29d-4e32b043f712"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:40:30.642276 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.642250    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98d0f11c-f25d-4960-b19c-1634f6b4e202-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98d0f11c-f25d-4960-b19c-1634f6b4e202" (UID: "98d0f11c-f25d-4960-b19c-1634f6b4e202"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:40:30.642387 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.642290    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98d0f11c-f25d-4960-b19c-1634f6b4e202-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "98d0f11c-f25d-4960-b19c-1634f6b4e202" (UID: "98d0f11c-f25d-4960-b19c-1634f6b4e202"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:40:30.642716 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.642699    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a8f26c8-acde-4d38-b29d-4e32b043f712-kube-api-access-c6pw6" (OuterVolumeSpecName: "kube-api-access-c6pw6") pod "2a8f26c8-acde-4d38-b29d-4e32b043f712" (UID: "2a8f26c8-acde-4d38-b29d-4e32b043f712"). InnerVolumeSpecName "kube-api-access-c6pw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:40:30.642761 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.642722    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98d0f11c-f25d-4960-b19c-1634f6b4e202-kube-api-access-vn78f" (OuterVolumeSpecName: "kube-api-access-vn78f") pod "98d0f11c-f25d-4960-b19c-1634f6b4e202" (UID: "98d0f11c-f25d-4960-b19c-1634f6b4e202"). InnerVolumeSpecName "kube-api-access-vn78f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:40:30.741671 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.741578    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98d0f11c-f25d-4960-b19c-1634f6b4e202-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:40:30.741671 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.741611    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vn78f\" (UniqueName: \"kubernetes.io/projected/98d0f11c-f25d-4960-b19c-1634f6b4e202-kube-api-access-vn78f\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:40:30.741671 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.741624    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98d0f11c-f25d-4960-b19c-1634f6b4e202-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:40:30.741671 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.741636    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6pw6\" (UniqueName: \"kubernetes.io/projected/2a8f26c8-acde-4d38-b29d-4e32b043f712-kube-api-access-c6pw6\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:40:30.741671 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.741648    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a8f26c8-acde-4d38-b29d-4e32b043f712-plugins-config-volume\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:40:30.943964 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:30.943933    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:31.062488 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.062385    2577 generic.go:356] "Generic (PLEG): container finished" podID="98d0f11c-f25d-4960-b19c-1634f6b4e202" containerID="3f5cc503e3697ac709031372aec8f8d96e819e80d0d8b0d113becf217d8da880" exitCode=0
Sep 07 10:40:31.062488 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.062446    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"
Sep 07 10:40:31.062719 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.062475    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6" event={"ID":"98d0f11c-f25d-4960-b19c-1634f6b4e202","Type":"ContainerDied","Data":"3f5cc503e3697ac709031372aec8f8d96e819e80d0d8b0d113becf217d8da880"}
Sep 07 10:40:31.062719 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.062549    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6" event={"ID":"98d0f11c-f25d-4960-b19c-1634f6b4e202","Type":"ContainerDied","Data":"45920f9612c28534b9fa3218dafce339a78d62e9623330bf6da8630ee4dafdd1"}
Sep 07 10:40:31.062719 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.062583    2577 scope.go:117] "RemoveContainer" containerID="3f5cc503e3697ac709031372aec8f8d96e819e80d0d8b0d113becf217d8da880"
Sep 07 10:40:31.064058 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.064033    2577 generic.go:356] "Generic (PLEG): container finished" podID="2a8f26c8-acde-4d38-b29d-4e32b043f712" containerID="13f97aa69149a1ea1fcbaf4f77310eb16035e88efbab0e05b7db73726085c813" exitCode=0
Sep 07 10:40:31.064212 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.064069    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"
Sep 07 10:40:31.064212 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.064111    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8" event={"ID":"2a8f26c8-acde-4d38-b29d-4e32b043f712","Type":"ContainerDied","Data":"13f97aa69149a1ea1fcbaf4f77310eb16035e88efbab0e05b7db73726085c813"}
Sep 07 10:40:31.064212 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.064174    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8" event={"ID":"2a8f26c8-acde-4d38-b29d-4e32b043f712","Type":"ContainerDied","Data":"a229425180796a2985fcaf1263f4b96c38b00af02b25b44d1f39ec17d3240d8c"}
Sep 07 10:40:31.073343 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.073277    2577 scope.go:117] "RemoveContainer" containerID="3f5cc503e3697ac709031372aec8f8d96e819e80d0d8b0d113becf217d8da880"
Sep 07 10:40:31.073602 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:40:31.073581    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f5cc503e3697ac709031372aec8f8d96e819e80d0d8b0d113becf217d8da880\": container with ID starting with 3f5cc503e3697ac709031372aec8f8d96e819e80d0d8b0d113becf217d8da880 not found: ID does not exist" containerID="3f5cc503e3697ac709031372aec8f8d96e819e80d0d8b0d113becf217d8da880"
Sep 07 10:40:31.073669 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.073612    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f5cc503e3697ac709031372aec8f8d96e819e80d0d8b0d113becf217d8da880"} err="failed to get container status \"3f5cc503e3697ac709031372aec8f8d96e819e80d0d8b0d113becf217d8da880\": rpc error: code = NotFound desc = could not find container \"3f5cc503e3697ac709031372aec8f8d96e819e80d0d8b0d113becf217d8da880\": container with ID starting with 3f5cc503e3697ac709031372aec8f8d96e819e80d0d8b0d113becf217d8da880 not found: ID does not exist"
Sep 07 10:40:31.073669 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.073627    2577 scope.go:117] "RemoveContainer" containerID="13f97aa69149a1ea1fcbaf4f77310eb16035e88efbab0e05b7db73726085c813"
Sep 07 10:40:31.081330 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.081304    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"]
Sep 07 10:40:31.087525 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.087504    2577 scope.go:117] "RemoveContainer" containerID="13f97aa69149a1ea1fcbaf4f77310eb16035e88efbab0e05b7db73726085c813"
Sep 07 10:40:31.087863 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:40:31.087844    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13f97aa69149a1ea1fcbaf4f77310eb16035e88efbab0e05b7db73726085c813\": container with ID starting with 13f97aa69149a1ea1fcbaf4f77310eb16035e88efbab0e05b7db73726085c813 not found: ID does not exist" containerID="13f97aa69149a1ea1fcbaf4f77310eb16035e88efbab0e05b7db73726085c813"
Sep 07 10:40:31.087959 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.087871    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13f97aa69149a1ea1fcbaf4f77310eb16035e88efbab0e05b7db73726085c813"} err="failed to get container status \"13f97aa69149a1ea1fcbaf4f77310eb16035e88efbab0e05b7db73726085c813\": rpc error: code = NotFound desc = could not find container \"13f97aa69149a1ea1fcbaf4f77310eb16035e88efbab0e05b7db73726085c813\": container with ID starting with 13f97aa69149a1ea1fcbaf4f77310eb16035e88efbab0e05b7db73726085c813 not found: ID does not exist"
Sep 07 10:40:31.088478 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.088459    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c5b29a-85dd59d946-mq5q8"]
Sep 07 10:40:31.099348 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.099328    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"]
Sep 07 10:40:31.102820 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:31.102795    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c5b29a-848d854ccb-lnhq6"]
Sep 07 10:40:32.785112 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:32.785070    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a8f26c8-acde-4d38-b29d-4e32b043f712" path="/var/lib/kubelet/pods/2a8f26c8-acde-4d38-b29d-4e32b043f712/volumes"
Sep 07 10:40:32.785506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:32.785492    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98d0f11c-f25d-4960-b19c-1634f6b4e202" path="/var/lib/kubelet/pods/98d0f11c-f25d-4960-b19c-1634f6b4e202/volumes"
Sep 07 10:40:33.209034 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:33.208999    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:36.068608 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:36.068564    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:42.260184 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:42.260124    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:42.748226 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:42.748187    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"]
Sep 07 10:40:42.748497 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:42.748466    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf" podUID="11276480-897e-47df-b89d-931ec05f2afe" containerName="maas-api" containerID="cri-o://310d09782ff31b9f2de85de5d942279f02488f8f44b77b87421185242907b7be" gracePeriod=30
Sep 07 10:40:42.776693 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:42.776656    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:42.879107 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:42.879072    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:42.979098 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:42.979060    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:43.009554 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.009482    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:43.065120 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.065075    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11276480-897e-47df-b89d-931ec05f2afe-openshift-service-ca\") pod \"11276480-897e-47df-b89d-931ec05f2afe\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") "
Sep 07 10:40:43.065316 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.065178    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11276480-897e-47df-b89d-931ec05f2afe-maas-api-tls\") pod \"11276480-897e-47df-b89d-931ec05f2afe\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") "
Sep 07 10:40:43.065316 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.065263    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-567ph\" (UniqueName: \"kubernetes.io/projected/11276480-897e-47df-b89d-931ec05f2afe-kube-api-access-567ph\") pod \"11276480-897e-47df-b89d-931ec05f2afe\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") "
Sep 07 10:40:43.065408 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.065387    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11276480-897e-47df-b89d-931ec05f2afe-metrics-certs\") pod \"11276480-897e-47df-b89d-931ec05f2afe\" (UID: \"11276480-897e-47df-b89d-931ec05f2afe\") "
Sep 07 10:40:43.065571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.065551    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11276480-897e-47df-b89d-931ec05f2afe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "11276480-897e-47df-b89d-931ec05f2afe" (UID: "11276480-897e-47df-b89d-931ec05f2afe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:40:43.065688 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.065672    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11276480-897e-47df-b89d-931ec05f2afe-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:40:43.067441 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.067408    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11276480-897e-47df-b89d-931ec05f2afe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "11276480-897e-47df-b89d-931ec05f2afe" (UID: "11276480-897e-47df-b89d-931ec05f2afe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:40:43.067601 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.067575    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11276480-897e-47df-b89d-931ec05f2afe-kube-api-access-567ph" (OuterVolumeSpecName: "kube-api-access-567ph") pod "11276480-897e-47df-b89d-931ec05f2afe" (UID: "11276480-897e-47df-b89d-931ec05f2afe"). InnerVolumeSpecName "kube-api-access-567ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:40:43.067666 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.067607    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11276480-897e-47df-b89d-931ec05f2afe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "11276480-897e-47df-b89d-931ec05f2afe" (UID: "11276480-897e-47df-b89d-931ec05f2afe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:40:43.077529 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.077501    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:43.141964 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.141925    2577 generic.go:356] "Generic (PLEG): container finished" podID="11276480-897e-47df-b89d-931ec05f2afe" containerID="310d09782ff31b9f2de85de5d942279f02488f8f44b77b87421185242907b7be" exitCode=0
Sep 07 10:40:43.142196 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.141979    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"
Sep 07 10:40:43.142196 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.142012    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf" event={"ID":"11276480-897e-47df-b89d-931ec05f2afe","Type":"ContainerDied","Data":"310d09782ff31b9f2de85de5d942279f02488f8f44b77b87421185242907b7be"}
Sep 07 10:40:43.142196 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.142069    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf" event={"ID":"11276480-897e-47df-b89d-931ec05f2afe","Type":"ContainerDied","Data":"2a0a51c55a0719db8203856a912cb49ce6e96de617545f26dfea705823673672"}
Sep 07 10:40:43.142196 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.142091    2577 scope.go:117] "RemoveContainer" containerID="310d09782ff31b9f2de85de5d942279f02488f8f44b77b87421185242907b7be"
Sep 07 10:40:43.152153 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.152119    2577 scope.go:117] "RemoveContainer" containerID="310d09782ff31b9f2de85de5d942279f02488f8f44b77b87421185242907b7be"
Sep 07 10:40:43.152435 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:40:43.152415    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"310d09782ff31b9f2de85de5d942279f02488f8f44b77b87421185242907b7be\": container with ID starting with 310d09782ff31b9f2de85de5d942279f02488f8f44b77b87421185242907b7be not found: ID does not exist" containerID="310d09782ff31b9f2de85de5d942279f02488f8f44b77b87421185242907b7be"
Sep 07 10:40:43.152498 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.152448    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"310d09782ff31b9f2de85de5d942279f02488f8f44b77b87421185242907b7be"} err="failed to get container status \"310d09782ff31b9f2de85de5d942279f02488f8f44b77b87421185242907b7be\": rpc error: code = NotFound desc = could not find container \"310d09782ff31b9f2de85de5d942279f02488f8f44b77b87421185242907b7be\": container with ID starting with 310d09782ff31b9f2de85de5d942279f02488f8f44b77b87421185242907b7be not found: ID does not exist"
Sep 07 10:40:43.166173 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.166120    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11276480-897e-47df-b89d-931ec05f2afe-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:40:43.166173 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.166174    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11276480-897e-47df-b89d-931ec05f2afe-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:40:43.166331 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.166186    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-567ph\" (UniqueName: \"kubernetes.io/projected/11276480-897e-47df-b89d-931ec05f2afe-kube-api-access-567ph\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:40:43.166622 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.166602    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"]
Sep 07 10:40:43.170360 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:43.170337    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc9185fe-7f45c69f94-sdrbf"]
Sep 07 10:40:44.492281 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:44.492253    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:44.785172 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:44.785062    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11276480-897e-47df-b89d-931ec05f2afe" path="/var/lib/kubelet/pods/11276480-897e-47df-b89d-931ec05f2afe/volumes"
Sep 07 10:40:45.023031 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:45.022993    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:45.389275 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:45.389241    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:45.746332 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:45.746299    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:40:45.756003 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:45.755970    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:47.333364 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:47.333331    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:48.143203 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:48.143163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:48.254498 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:48.254459    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:49.466360 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.466329    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"]
Sep 07 10:40:49.466823 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.466722    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98d0f11c-f25d-4960-b19c-1634f6b4e202" containerName="maas-api"
Sep 07 10:40:49.466823 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.466732    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="98d0f11c-f25d-4960-b19c-1634f6b4e202" containerName="maas-api"
Sep 07 10:40:49.466823 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.466745    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11276480-897e-47df-b89d-931ec05f2afe" containerName="maas-api"
Sep 07 10:40:49.466823 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.466750    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="11276480-897e-47df-b89d-931ec05f2afe" containerName="maas-api"
Sep 07 10:40:49.466823 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.466759    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a8f26c8-acde-4d38-b29d-4e32b043f712" containerName="payload-pre-processing"
Sep 07 10:40:49.466823 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.466764    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a8f26c8-acde-4d38-b29d-4e32b043f712" containerName="payload-pre-processing"
Sep 07 10:40:49.467185 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.466841    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a8f26c8-acde-4d38-b29d-4e32b043f712" containerName="payload-pre-processing"
Sep 07 10:40:49.467185 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.466850    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="11276480-897e-47df-b89d-931ec05f2afe" containerName="maas-api"
Sep 07 10:40:49.467185 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.466858    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="98d0f11c-f25d-4960-b19c-1634f6b4e202" containerName="maas-api"
Sep 07 10:40:49.471064 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.471038    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:49.473606 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.473579    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-1ccb3bcb-serving-cert\""
Sep 07 10:40:49.482975 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.482949    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"]
Sep 07 10:40:49.505392 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.505342    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:49.526472 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.526442    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d4670ca-e123-4888-9c37-9dc9cb2ca894-openshift-service-ca\") pod \"maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:49.526630 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.526478    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d4670ca-e123-4888-9c37-9dc9cb2ca894-maas-api-tls\") pod \"maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:49.526630 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.526551    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4670ca-e123-4888-9c37-9dc9cb2ca894-metrics-certs\") pod \"maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:49.526630 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.526607    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvzcb\" (UniqueName: \"kubernetes.io/projected/8d4670ca-e123-4888-9c37-9dc9cb2ca894-kube-api-access-zvzcb\") pod \"maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:49.614798 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.614763    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:49.627855 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.627828    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d4670ca-e123-4888-9c37-9dc9cb2ca894-openshift-service-ca\") pod \"maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:49.627996 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.627863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d4670ca-e123-4888-9c37-9dc9cb2ca894-maas-api-tls\") pod \"maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:49.627996 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.627923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4670ca-e123-4888-9c37-9dc9cb2ca894-metrics-certs\") pod \"maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:49.628071 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:40:49.628047    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-serving-cert: secret "maas-api-e2e-derive-1ccb3bcb-serving-cert" not found
Sep 07 10:40:49.628164 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:40:49.628123    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d4670ca-e123-4888-9c37-9dc9cb2ca894-maas-api-tls podName:8d4670ca-e123-4888-9c37-9dc9cb2ca894 nodeName:}" failed. No retries permitted until 2026-09-07 10:40:50.128103168 +0000 UTC m=+1253.862277380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8d4670ca-e123-4888-9c37-9dc9cb2ca894-maas-api-tls") pod "maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84" (UID: "8d4670ca-e123-4888-9c37-9dc9cb2ca894") : secret "maas-api-e2e-derive-1ccb3bcb-serving-cert" not found
Sep 07 10:40:49.628230 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.628179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvzcb\" (UniqueName: \"kubernetes.io/projected/8d4670ca-e123-4888-9c37-9dc9cb2ca894-kube-api-access-zvzcb\") pod \"maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:49.628643 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.628623    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d4670ca-e123-4888-9c37-9dc9cb2ca894-openshift-service-ca\") pod \"maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:49.630531 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.630515    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4670ca-e123-4888-9c37-9dc9cb2ca894-metrics-certs\") pod \"maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:49.638484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.638466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvzcb\" (UniqueName: \"kubernetes.io/projected/8d4670ca-e123-4888-9c37-9dc9cb2ca894-kube-api-access-zvzcb\") pod \"maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:49.981582 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:49.981550    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:50.077541 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:50.077485    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:50.132360 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:50.132323    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d4670ca-e123-4888-9c37-9dc9cb2ca894-maas-api-tls\") pod \"maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:50.135337 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:50.135308    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d4670ca-e123-4888-9c37-9dc9cb2ca894-maas-api-tls\") pod \"maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:50.156402 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:50.156367    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:50.385018 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:50.384922    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:50.659211 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:50.659179    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"]
Sep 07 10:40:50.660877 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:40:50.660849    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d4670ca_e123_4888_9c37_9dc9cb2ca894.slice/crio-27522fa593453a22c9105a8d6e8bae510986d243826be6ce5704c6726db7fc29 WatchSource:0}: Error finding container 27522fa593453a22c9105a8d6e8bae510986d243826be6ce5704c6726db7fc29: Status 404 returned error can't find the container with id 27522fa593453a22c9105a8d6e8bae510986d243826be6ce5704c6726db7fc29
Sep 07 10:40:51.193401 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:51.193368    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84" event={"ID":"8d4670ca-e123-4888-9c37-9dc9cb2ca894","Type":"ContainerStarted","Data":"6c2e05b407e702f312bd9abf81c8ac0e93b41ebac8e910f03ccfb4f9d5fc4fe5"}
Sep 07 10:40:51.193596 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:51.193419    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84" event={"ID":"8d4670ca-e123-4888-9c37-9dc9cb2ca894","Type":"ContainerStarted","Data":"27522fa593453a22c9105a8d6e8bae510986d243826be6ce5704c6726db7fc29"}
Sep 07 10:40:51.193596 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:51.193451    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:40:51.211491 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:51.211445    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84" podStartSLOduration=1.831675901 podStartE2EDuration="2.211427094s" totalImagesPullingTime="379.751193ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:40:51.210839621 +0000 UTC m=+1254.945013854" watchObservedRunningTime="2026-09-07 10:40:51.211427094 +0000 UTC m=+1254.945601383"
Sep 07 10:40:54.539751 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:54.539720    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:40:57.206779 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:40:57.206749    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:41:03.841210 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:03.841168    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:41:09.972511 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:09.972466    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:10.241950 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:10.241789    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:10.788270 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:10.788226    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:11.159475 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:11.159392    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:11.227607 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:11.227575    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:14.199488 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:14.199437    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:14.290814 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:14.290781    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:15.307301 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.307271    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"]
Sep 07 10:41:15.312837 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.312810    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:15.315552 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.315522    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-a49ef7-serving-cert\""
Sep 07 10:41:15.322464 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.322436    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"]
Sep 07 10:41:15.347983 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.347954    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:15.370743 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.370687    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qcv8\" (UniqueName: \"kubernetes.io/projected/ca03acb1-c581-4098-87e4-5c3c18768909-kube-api-access-4qcv8\") pod \"maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:15.370904 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.370790    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca03acb1-c581-4098-87e4-5c3c18768909-openshift-service-ca\") pod \"maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:15.370904 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.370882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-maas-api-tls\") pod \"maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:15.370904 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.370902    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-metrics-certs\") pod \"maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:15.448489 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.448451    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:15.471797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.471765    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-maas-api-tls\") pod \"maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:15.471797 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.471799    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-metrics-certs\") pod \"maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:15.472068 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.471855    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qcv8\" (UniqueName: \"kubernetes.io/projected/ca03acb1-c581-4098-87e4-5c3c18768909-kube-api-access-4qcv8\") pod \"maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:15.472068 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.471916    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca03acb1-c581-4098-87e4-5c3c18768909-openshift-service-ca\") pod \"maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:15.472068 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:41:15.471938    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-serving-cert: secret "maas-api-e2e-explicit-a49ef7-serving-cert" not found
Sep 07 10:41:15.472068 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:41:15.472066    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-maas-api-tls podName:ca03acb1-c581-4098-87e4-5c3c18768909 nodeName:}" failed. No retries permitted until 2026-09-07 10:41:15.972045401 +0000 UTC m=+1279.706219626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-maas-api-tls") pod "maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv" (UID: "ca03acb1-c581-4098-87e4-5c3c18768909") : secret "maas-api-e2e-explicit-a49ef7-serving-cert" not found
Sep 07 10:41:15.473240 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.473206    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca03acb1-c581-4098-87e4-5c3c18768909-openshift-service-ca\") pod \"maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:15.474945 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.474919    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-metrics-certs\") pod \"maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:15.483303 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.483278    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qcv8\" (UniqueName: \"kubernetes.io/projected/ca03acb1-c581-4098-87e4-5c3c18768909-kube-api-access-4qcv8\") pod \"maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:15.878729 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.878696    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:15.961435 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.961390    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:15.977606 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:15.977525    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-maas-api-tls\") pod \"maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:15.977811 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:41:15.977735    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-serving-cert: secret "maas-api-e2e-explicit-a49ef7-serving-cert" not found
Sep 07 10:41:15.977883 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:41:15.977821    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-maas-api-tls podName:ca03acb1-c581-4098-87e4-5c3c18768909 nodeName:}" failed. No retries permitted until 2026-09-07 10:41:16.977794499 +0000 UTC m=+1280.711968719 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-maas-api-tls") pod "maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv" (UID: "ca03acb1-c581-4098-87e4-5c3c18768909") : secret "maas-api-e2e-explicit-a49ef7-serving-cert" not found
Sep 07 10:41:16.105385 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:16.105352    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:16.399745 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:16.399710    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:16.533684 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:16.533641    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:16.987660 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:16.987623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-maas-api-tls\") pod \"maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:16.990515 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:16.990492    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-maas-api-tls\") pod \"maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:17.125694 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:17.125664    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:17.207393 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:17.207355    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:17.419149 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:17.419091    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"]
Sep 07 10:41:17.420171 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:41:17.420117    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca03acb1_c581_4098_87e4_5c3c18768909.slice/crio-c768491a8346752b2e3c2c14f0ff18b62f22e8e4361ebebfbf94af9d7b32eb52 WatchSource:0}: Error finding container c768491a8346752b2e3c2c14f0ff18b62f22e8e4361ebebfbf94af9d7b32eb52: Status 404 returned error can't find the container with id c768491a8346752b2e3c2c14f0ff18b62f22e8e4361ebebfbf94af9d7b32eb52
Sep 07 10:41:17.421543 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:17.421525    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 10:41:18.353859 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:18.353826    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv" event={"ID":"ca03acb1-c581-4098-87e4-5c3c18768909","Type":"ContainerStarted","Data":"1483b5564fb3d840e6a317d7ee88d2c6906201507f49c20d677cb45bee1135c4"}
Sep 07 10:41:18.354035 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:18.353884    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv" event={"ID":"ca03acb1-c581-4098-87e4-5c3c18768909","Type":"ContainerStarted","Data":"c768491a8346752b2e3c2c14f0ff18b62f22e8e4361ebebfbf94af9d7b32eb52"}
Sep 07 10:41:18.354035 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:18.354017    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:18.374350 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:18.374284    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv" podStartSLOduration=3.009656133 podStartE2EDuration="3.374261327s" totalImagesPullingTime="364.605194ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:41:18.371838754 +0000 UTC m=+1282.106013011" watchObservedRunningTime="2026-09-07 10:41:18.374261327 +0000 UTC m=+1282.108435563"
Sep 07 10:41:19.503088 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:19.503050    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:20.162418 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.162383    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"]
Sep 07 10:41:20.162655 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.162618    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84" podUID="8d4670ca-e123-4888-9c37-9dc9cb2ca894" containerName="maas-api" containerID="cri-o://6c2e05b407e702f312bd9abf81c8ac0e93b41ebac8e910f03ccfb4f9d5fc4fe5" gracePeriod=30
Sep 07 10:41:20.188191 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.188155    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:20.257560 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.257518    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:20.356998 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.356949    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:20.368835 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.368800    2577 generic.go:356] "Generic (PLEG): container finished" podID="8d4670ca-e123-4888-9c37-9dc9cb2ca894" containerID="6c2e05b407e702f312bd9abf81c8ac0e93b41ebac8e910f03ccfb4f9d5fc4fe5" exitCode=0
Sep 07 10:41:20.369039 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.368847    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84" event={"ID":"8d4670ca-e123-4888-9c37-9dc9cb2ca894","Type":"ContainerDied","Data":"6c2e05b407e702f312bd9abf81c8ac0e93b41ebac8e910f03ccfb4f9d5fc4fe5"}
Sep 07 10:41:20.422400 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.422304    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:20.425813 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.425792    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:41:20.522096 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.522064    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d4670ca-e123-4888-9c37-9dc9cb2ca894-openshift-service-ca\") pod \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") "
Sep 07 10:41:20.522591 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.522172    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d4670ca-e123-4888-9c37-9dc9cb2ca894-maas-api-tls\") pod \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") "
Sep 07 10:41:20.522591 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.522228    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4670ca-e123-4888-9c37-9dc9cb2ca894-metrics-certs\") pod \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") "
Sep 07 10:41:20.522591 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.522305    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvzcb\" (UniqueName: \"kubernetes.io/projected/8d4670ca-e123-4888-9c37-9dc9cb2ca894-kube-api-access-zvzcb\") pod \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\" (UID: \"8d4670ca-e123-4888-9c37-9dc9cb2ca894\") "
Sep 07 10:41:20.522591 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.522473    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d4670ca-e123-4888-9c37-9dc9cb2ca894-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8d4670ca-e123-4888-9c37-9dc9cb2ca894" (UID: "8d4670ca-e123-4888-9c37-9dc9cb2ca894"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:41:20.522823 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.522646    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d4670ca-e123-4888-9c37-9dc9cb2ca894-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:41:20.526086 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.526048    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d4670ca-e123-4888-9c37-9dc9cb2ca894-kube-api-access-zvzcb" (OuterVolumeSpecName: "kube-api-access-zvzcb") pod "8d4670ca-e123-4888-9c37-9dc9cb2ca894" (UID: "8d4670ca-e123-4888-9c37-9dc9cb2ca894"). InnerVolumeSpecName "kube-api-access-zvzcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:41:20.526336 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.526307    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d4670ca-e123-4888-9c37-9dc9cb2ca894-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8d4670ca-e123-4888-9c37-9dc9cb2ca894" (UID: "8d4670ca-e123-4888-9c37-9dc9cb2ca894"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:41:20.526611 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.526592    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d4670ca-e123-4888-9c37-9dc9cb2ca894-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8d4670ca-e123-4888-9c37-9dc9cb2ca894" (UID: "8d4670ca-e123-4888-9c37-9dc9cb2ca894"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:41:20.556706 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.556671    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:20.612062 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.612025    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:20.623729 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.623705    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvzcb\" (UniqueName: \"kubernetes.io/projected/8d4670ca-e123-4888-9c37-9dc9cb2ca894-kube-api-access-zvzcb\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:41:20.623729 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.623729    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d4670ca-e123-4888-9c37-9dc9cb2ca894-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:41:20.623889 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:20.623739    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d4670ca-e123-4888-9c37-9dc9cb2ca894-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:41:21.359093 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:21.359055    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:21.379747 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:21.379718    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"
Sep 07 10:41:21.379911 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:21.379764    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84" event={"ID":"8d4670ca-e123-4888-9c37-9dc9cb2ca894","Type":"ContainerDied","Data":"27522fa593453a22c9105a8d6e8bae510986d243826be6ce5704c6726db7fc29"}
Sep 07 10:41:21.379911 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:21.379823    2577 scope.go:117] "RemoveContainer" containerID="6c2e05b407e702f312bd9abf81c8ac0e93b41ebac8e910f03ccfb4f9d5fc4fe5"
Sep 07 10:41:21.405675 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:21.405640    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"]
Sep 07 10:41:21.410374 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:21.410335    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1ccb3bcb-85b4b68845-psv84"]
Sep 07 10:41:21.456647 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:21.456603    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:21.562585 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:21.562547    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:22.785423 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:22.785380    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d4670ca-e123-4888-9c37-9dc9cb2ca894" path="/var/lib/kubelet/pods/8d4670ca-e123-4888-9c37-9dc9cb2ca894/volumes"
Sep 07 10:41:23.179974 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:23.179881    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:24.346661 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:24.346625    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:41:24.365845 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:24.365812    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:41:25.130538 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.130502    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"]
Sep 07 10:41:25.130985 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.130966    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d4670ca-e123-4888-9c37-9dc9cb2ca894" containerName="maas-api"
Sep 07 10:41:25.131048 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.130990    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d4670ca-e123-4888-9c37-9dc9cb2ca894" containerName="maas-api"
Sep 07 10:41:25.131101 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.131091    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d4670ca-e123-4888-9c37-9dc9cb2ca894" containerName="maas-api"
Sep 07 10:41:25.134387 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.134365    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.136886 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.136864    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b4061f\"/\"kube-root-ca.crt\""
Sep 07 10:41:25.137030 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.136938    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b4061f\"/\"default-dockercfg-tqkp8\""
Sep 07 10:41:25.138066 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.138044    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b4061f\"/\"test-model-b4061f-kserve-self-signed-certs\""
Sep 07 10:41:25.138474 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.138374    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b4061f\"/\"openshift-service-ca.crt\""
Sep 07 10:41:25.141565 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.141543    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"]
Sep 07 10:41:25.159659 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.159617    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:25.271047 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.271001    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-dshm\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.271301 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.271053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-tmp-dir\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.271301 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.271211    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mdm6\" (UniqueName: \"kubernetes.io/projected/072d592f-757b-4f20-a43f-886db4fdbba2-kube-api-access-8mdm6\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.271301 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.271287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-model-cache\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.271493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.271336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/072d592f-757b-4f20-a43f-886db4fdbba2-tls-certs\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.271493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.271366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-home\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.372631 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.372585    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-tmp-dir\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.373184 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.372669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mdm6\" (UniqueName: \"kubernetes.io/projected/072d592f-757b-4f20-a43f-886db4fdbba2-kube-api-access-8mdm6\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.373184 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.372760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-model-cache\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.373184 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.372797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/072d592f-757b-4f20-a43f-886db4fdbba2-tls-certs\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.373184 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.372825    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-home\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.373184 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.372892    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-dshm\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.373184 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.373054    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-tmp-dir\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.373620 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.373583    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-home\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.374053 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.374029    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-model-cache\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.375899 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.375873    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-dshm\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.376178 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.376159    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/072d592f-757b-4f20-a43f-886db4fdbba2-tls-certs\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.384032 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.383966    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mdm6\" (UniqueName: \"kubernetes.io/projected/072d592f-757b-4f20-a43f-886db4fdbba2-kube-api-access-8mdm6\") pod \"test-model-b4061f-kserve-687465d747-rz4wn\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") " pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.446381 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.446344    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:25.728576 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:25.728549    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"]
Sep 07 10:41:25.730386 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:41:25.730351    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod072d592f_757b_4f20_a43f_886db4fdbba2.slice/crio-3b4af272dc15730cfd14dcbf81769ff2c298d472307932b6fc1351dddb61daf1 WatchSource:0}: Error finding container 3b4af272dc15730cfd14dcbf81769ff2c298d472307932b6fc1351dddb61daf1: Status 404 returned error can't find the container with id 3b4af272dc15730cfd14dcbf81769ff2c298d472307932b6fc1351dddb61daf1
Sep 07 10:41:26.417142 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:26.417099    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn" event={"ID":"072d592f-757b-4f20-a43f-886db4fdbba2","Type":"ContainerStarted","Data":"b2b6b0a6d6cdaa1e42db3ffd4a99e13d6398fb6035800fda722e375fdb7b5c14"}
Sep 07 10:41:26.417555 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:26.417159    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn" event={"ID":"072d592f-757b-4f20-a43f-886db4fdbba2","Type":"ContainerStarted","Data":"3b4af272dc15730cfd14dcbf81769ff2c298d472307932b6fc1351dddb61daf1"}
Sep 07 10:41:26.435488 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:26.435431    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn" podStartSLOduration=1.435413443 podStartE2EDuration="1.435413443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:41:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:41:26.43312058 +0000 UTC m=+1290.167294824" watchObservedRunningTime="2026-09-07 10:41:26.435413443 +0000 UTC m=+1290.169587675"
Sep 07 10:41:29.208481 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:29.208449    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:29.318219 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:29.318186    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:29.435293 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:29.435256    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:29.569388 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:29.569263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:29.986841 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:29.986801    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:35.447034 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:35.446995    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:35.447515 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:35.447069    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:35.459462 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:35.459435    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:35.490050 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:35.490018    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:41:36.362005 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:36.361966    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:37.839270 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:37.839237    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:39.337849 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:39.337762    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:43.159643 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:43.159594    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:43.635465 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:43.635430    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:41:55.961601 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:55.961561    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:57.201992 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:57.201953    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:41:59.320846 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:41:59.320809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:01.195924 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:01.195890    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:42:01.931424 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:01.931382    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:02.034704 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:02.034660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:02.131537 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:02.131503    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:09.891992 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:09.891949    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"]
Sep 07 10:42:09.892421 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:09.892187    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv" podUID="ca03acb1-c581-4098-87e4-5c3c18768909" containerName="maas-api" containerID="cri-o://1483b5564fb3d840e6a317d7ee88d2c6906201507f49c20d677cb45bee1135c4" gracePeriod=30
Sep 07 10:42:09.919684 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:09.919652    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:10.020492 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.020457    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:10.119080 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.119041    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:10.159175 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.159148    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:42:10.178056 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.178025    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qcv8\" (UniqueName: \"kubernetes.io/projected/ca03acb1-c581-4098-87e4-5c3c18768909-kube-api-access-4qcv8\") pod \"ca03acb1-c581-4098-87e4-5c3c18768909\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") "
Sep 07 10:42:10.178380 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.178357    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca03acb1-c581-4098-87e4-5c3c18768909-openshift-service-ca\") pod \"ca03acb1-c581-4098-87e4-5c3c18768909\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") "
Sep 07 10:42:10.178567 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.178550    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-metrics-certs\") pod \"ca03acb1-c581-4098-87e4-5c3c18768909\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") "
Sep 07 10:42:10.178818 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.178791    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-maas-api-tls\") pod \"ca03acb1-c581-4098-87e4-5c3c18768909\" (UID: \"ca03acb1-c581-4098-87e4-5c3c18768909\") "
Sep 07 10:42:10.179271 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.178891    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca03acb1-c581-4098-87e4-5c3c18768909-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca03acb1-c581-4098-87e4-5c3c18768909" (UID: "ca03acb1-c581-4098-87e4-5c3c18768909"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:42:10.179479 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.179460    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca03acb1-c581-4098-87e4-5c3c18768909-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:42:10.182028 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.179876    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:10.182864 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.182438    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ca03acb1-c581-4098-87e4-5c3c18768909" (UID: "ca03acb1-c581-4098-87e4-5c3c18768909"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:42:10.182864 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.182767    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca03acb1-c581-4098-87e4-5c3c18768909-kube-api-access-4qcv8" (OuterVolumeSpecName: "kube-api-access-4qcv8") pod "ca03acb1-c581-4098-87e4-5c3c18768909" (UID: "ca03acb1-c581-4098-87e4-5c3c18768909"). InnerVolumeSpecName "kube-api-access-4qcv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:42:10.183525 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.183453    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca03acb1-c581-4098-87e4-5c3c18768909" (UID: "ca03acb1-c581-4098-87e4-5c3c18768909"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:42:10.280419 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.280384    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qcv8\" (UniqueName: \"kubernetes.io/projected/ca03acb1-c581-4098-87e4-5c3c18768909-kube-api-access-4qcv8\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:42:10.280419 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.280415    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:42:10.280419 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.280426    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca03acb1-c581-4098-87e4-5c3c18768909-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:42:10.521512 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.521472    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:10.689362 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.689331    2577 generic.go:356] "Generic (PLEG): container finished" podID="ca03acb1-c581-4098-87e4-5c3c18768909" containerID="1483b5564fb3d840e6a317d7ee88d2c6906201507f49c20d677cb45bee1135c4" exitCode=0
Sep 07 10:42:10.689537 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.689378    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"
Sep 07 10:42:10.689537 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.689426    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv" event={"ID":"ca03acb1-c581-4098-87e4-5c3c18768909","Type":"ContainerDied","Data":"1483b5564fb3d840e6a317d7ee88d2c6906201507f49c20d677cb45bee1135c4"}
Sep 07 10:42:10.689537 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.689489    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv" event={"ID":"ca03acb1-c581-4098-87e4-5c3c18768909","Type":"ContainerDied","Data":"c768491a8346752b2e3c2c14f0ff18b62f22e8e4361ebebfbf94af9d7b32eb52"}
Sep 07 10:42:10.689537 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.689519    2577 scope.go:117] "RemoveContainer" containerID="1483b5564fb3d840e6a317d7ee88d2c6906201507f49c20d677cb45bee1135c4"
Sep 07 10:42:10.699982 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.699963    2577 scope.go:117] "RemoveContainer" containerID="1483b5564fb3d840e6a317d7ee88d2c6906201507f49c20d677cb45bee1135c4"
Sep 07 10:42:10.700340 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:42:10.700318    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1483b5564fb3d840e6a317d7ee88d2c6906201507f49c20d677cb45bee1135c4\": container with ID starting with 1483b5564fb3d840e6a317d7ee88d2c6906201507f49c20d677cb45bee1135c4 not found: ID does not exist" containerID="1483b5564fb3d840e6a317d7ee88d2c6906201507f49c20d677cb45bee1135c4"
Sep 07 10:42:10.700395 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.700351    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1483b5564fb3d840e6a317d7ee88d2c6906201507f49c20d677cb45bee1135c4"} err="failed to get container status \"1483b5564fb3d840e6a317d7ee88d2c6906201507f49c20d677cb45bee1135c4\": rpc error: code = NotFound desc = could not find container \"1483b5564fb3d840e6a317d7ee88d2c6906201507f49c20d677cb45bee1135c4\": container with ID starting with 1483b5564fb3d840e6a317d7ee88d2c6906201507f49c20d677cb45bee1135c4 not found: ID does not exist"
Sep 07 10:42:10.726944 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.726913    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"]
Sep 07 10:42:10.749093 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.749053    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a49ef7-5d54598ddf-dvrjv"]
Sep 07 10:42:10.786488 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:10.786404    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca03acb1-c581-4098-87e4-5c3c18768909" path="/var/lib/kubelet/pods/ca03acb1-c581-4098-87e4-5c3c18768909/volumes"
Sep 07 10:42:13.826636 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:13.826599    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:13.927904 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:13.927868    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:14.742970 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:14.742935    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:42:16.367413 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:16.367369    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:16.433793 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:16.433758    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:16.646184 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:16.646065    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:17.107602 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:17.107561    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:17.517605 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:17.517562    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:18.181617 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:18.181581    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:18.268696 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:18.268653    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:18.622202 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:18.622163    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:18.724971 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:18.724923    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:18.899517 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:18.899427    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:23.997520 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:23.997478    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:24.061875 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:24.061828    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:24.130995 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:24.130957    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:24.231344 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:24.231316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:25.168778 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:25.168741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:29.137476 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:29.137445    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:42:34.924675 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:34.924640    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:36.235828 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:36.235798    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:36.336929 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:36.336889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:37.123116 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:37.123080    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:37.275001 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:37.274961    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:37.525884 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:37.525845    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:37.669259 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:37.669218    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:38.407168 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:38.407117    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"]
Sep 07 10:42:38.430587 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:38.430527    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:38.694673 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:38.694584    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:42.344992 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:42.344955    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:42.444637 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:42.444601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:42.645808 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:42.645718    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:42.742182 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:42.742123    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:42.845310 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:42.845280    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:43.664827 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:43.664790    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:43.753057 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:43.753014    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:43.853740 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:43.853695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:45.496386 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:45.496353    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:46.338335 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:46.338296    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:42:49.119184 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.119146    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"]
Sep 07 10:42:49.119641 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.119417    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4" podUID="829d7efb-68ac-4b7f-af51-59d9d763633d" containerName="maas-api" containerID="cri-o://ef5ddb76220015be93f8860163027d2f79abae4b3ba5a1e2ec95fda08c709851" gracePeriod=30
Sep 07 10:42:49.144929 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.144901    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:49.217044 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.217007    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:49.285634 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.285597    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:49.357112 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.357077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:49.375846 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.375784    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:42:49.422067 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.422032    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:49.449908 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.449877    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/829d7efb-68ac-4b7f-af51-59d9d763633d-openshift-service-ca\") pod \"829d7efb-68ac-4b7f-af51-59d9d763633d\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") "
Sep 07 10:42:49.450058 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.449943    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829d7efb-68ac-4b7f-af51-59d9d763633d-maas-api-tls\") pod \"829d7efb-68ac-4b7f-af51-59d9d763633d\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") "
Sep 07 10:42:49.450058 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.449997    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/829d7efb-68ac-4b7f-af51-59d9d763633d-metrics-certs\") pod \"829d7efb-68ac-4b7f-af51-59d9d763633d\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") "
Sep 07 10:42:49.450058 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.450032    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7vhg\" (UniqueName: \"kubernetes.io/projected/829d7efb-68ac-4b7f-af51-59d9d763633d-kube-api-access-f7vhg\") pod \"829d7efb-68ac-4b7f-af51-59d9d763633d\" (UID: \"829d7efb-68ac-4b7f-af51-59d9d763633d\") "
Sep 07 10:42:49.450351 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.450330    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/829d7efb-68ac-4b7f-af51-59d9d763633d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "829d7efb-68ac-4b7f-af51-59d9d763633d" (UID: "829d7efb-68ac-4b7f-af51-59d9d763633d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:42:49.452268 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.452240    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/829d7efb-68ac-4b7f-af51-59d9d763633d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "829d7efb-68ac-4b7f-af51-59d9d763633d" (UID: "829d7efb-68ac-4b7f-af51-59d9d763633d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:42:49.452473 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.452322    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/829d7efb-68ac-4b7f-af51-59d9d763633d-kube-api-access-f7vhg" (OuterVolumeSpecName: "kube-api-access-f7vhg") pod "829d7efb-68ac-4b7f-af51-59d9d763633d" (UID: "829d7efb-68ac-4b7f-af51-59d9d763633d"). InnerVolumeSpecName "kube-api-access-f7vhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:42:49.452473 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.452333    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/829d7efb-68ac-4b7f-af51-59d9d763633d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "829d7efb-68ac-4b7f-af51-59d9d763633d" (UID: "829d7efb-68ac-4b7f-af51-59d9d763633d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:42:49.550889 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.550850    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829d7efb-68ac-4b7f-af51-59d9d763633d-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:42:49.550889 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.550882    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/829d7efb-68ac-4b7f-af51-59d9d763633d-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:42:49.550889 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.550896    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7vhg\" (UniqueName: \"kubernetes.io/projected/829d7efb-68ac-4b7f-af51-59d9d763633d-kube-api-access-f7vhg\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:42:49.551116 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.550909    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/829d7efb-68ac-4b7f-af51-59d9d763633d-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:42:49.735360 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.735324    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:49.926502 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.926468    2577 generic.go:356] "Generic (PLEG): container finished" podID="829d7efb-68ac-4b7f-af51-59d9d763633d" containerID="ef5ddb76220015be93f8860163027d2f79abae4b3ba5a1e2ec95fda08c709851" exitCode=0
Sep 07 10:42:49.926669 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.926507    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4" event={"ID":"829d7efb-68ac-4b7f-af51-59d9d763633d","Type":"ContainerDied","Data":"ef5ddb76220015be93f8860163027d2f79abae4b3ba5a1e2ec95fda08c709851"}
Sep 07 10:42:49.926669 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.926527    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"
Sep 07 10:42:49.926669 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.926569    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4" event={"ID":"829d7efb-68ac-4b7f-af51-59d9d763633d","Type":"ContainerDied","Data":"151c198fa4ab6565afa7a2dbac5916ba531b22e96945a3e686fadcf071448ada"}
Sep 07 10:42:49.926669 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.926592    2577 scope.go:117] "RemoveContainer" containerID="ef5ddb76220015be93f8860163027d2f79abae4b3ba5a1e2ec95fda08c709851"
Sep 07 10:42:49.936729 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.936711    2577 scope.go:117] "RemoveContainer" containerID="ef5ddb76220015be93f8860163027d2f79abae4b3ba5a1e2ec95fda08c709851"
Sep 07 10:42:49.936990 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:42:49.936968    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef5ddb76220015be93f8860163027d2f79abae4b3ba5a1e2ec95fda08c709851\": container with ID starting with ef5ddb76220015be93f8860163027d2f79abae4b3ba5a1e2ec95fda08c709851 not found: ID does not exist" containerID="ef5ddb76220015be93f8860163027d2f79abae4b3ba5a1e2ec95fda08c709851"
Sep 07 10:42:49.937051 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.936994    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef5ddb76220015be93f8860163027d2f79abae4b3ba5a1e2ec95fda08c709851"} err="failed to get container status \"ef5ddb76220015be93f8860163027d2f79abae4b3ba5a1e2ec95fda08c709851\": rpc error: code = NotFound desc = could not find container \"ef5ddb76220015be93f8860163027d2f79abae4b3ba5a1e2ec95fda08c709851\": container with ID starting with ef5ddb76220015be93f8860163027d2f79abae4b3ba5a1e2ec95fda08c709851 not found: ID does not exist"
Sep 07 10:42:49.949582 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.949554    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"]
Sep 07 10:42:49.953802 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:49.953777    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1ef687-654b5dd98f-h6qj4"]
Sep 07 10:42:50.785032 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:50.785001    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="829d7efb-68ac-4b7f-af51-59d9d763633d" path="/var/lib/kubelet/pods/829d7efb-68ac-4b7f-af51-59d9d763633d/volumes"
Sep 07 10:42:51.470662 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:51.470629    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:51.549072 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:51.549040    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:52.200022 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:52.199987    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:53.428610 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.428567    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn" podUID="072d592f-757b-4f20-a43f-886db4fdbba2" containerName="main" containerID="cri-o://b2b6b0a6d6cdaa1e42db3ffd4a99e13d6398fb6035800fda722e375fdb7b5c14" gracePeriod=45
Sep 07 10:42:53.673788 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.673758    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:42:53.792668 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.792637    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-home\") pod \"072d592f-757b-4f20-a43f-886db4fdbba2\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") "
Sep 07 10:42:53.792841 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.792684    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-dshm\") pod \"072d592f-757b-4f20-a43f-886db4fdbba2\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") "
Sep 07 10:42:53.792841 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.792709    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-tmp-dir\") pod \"072d592f-757b-4f20-a43f-886db4fdbba2\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") "
Sep 07 10:42:53.792841 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.792797    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/072d592f-757b-4f20-a43f-886db4fdbba2-tls-certs\") pod \"072d592f-757b-4f20-a43f-886db4fdbba2\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") "
Sep 07 10:42:53.793034 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.792844    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mdm6\" (UniqueName: \"kubernetes.io/projected/072d592f-757b-4f20-a43f-886db4fdbba2-kube-api-access-8mdm6\") pod \"072d592f-757b-4f20-a43f-886db4fdbba2\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") "
Sep 07 10:42:53.793034 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.792877    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-model-cache\") pod \"072d592f-757b-4f20-a43f-886db4fdbba2\" (UID: \"072d592f-757b-4f20-a43f-886db4fdbba2\") "
Sep 07 10:42:53.793034 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.792927    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-home" (OuterVolumeSpecName: "home") pod "072d592f-757b-4f20-a43f-886db4fdbba2" (UID: "072d592f-757b-4f20-a43f-886db4fdbba2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:42:53.793034 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.793014    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "072d592f-757b-4f20-a43f-886db4fdbba2" (UID: "072d592f-757b-4f20-a43f-886db4fdbba2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:42:53.793287 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.793161    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-home\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:42:53.793287 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.793182    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-tmp-dir\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:42:53.793287 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.793253    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-model-cache" (OuterVolumeSpecName: "model-cache") pod "072d592f-757b-4f20-a43f-886db4fdbba2" (UID: "072d592f-757b-4f20-a43f-886db4fdbba2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:42:53.795236 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.795210    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/072d592f-757b-4f20-a43f-886db4fdbba2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "072d592f-757b-4f20-a43f-886db4fdbba2" (UID: "072d592f-757b-4f20-a43f-886db4fdbba2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:42:53.795610 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.795578    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-dshm" (OuterVolumeSpecName: "dshm") pod "072d592f-757b-4f20-a43f-886db4fdbba2" (UID: "072d592f-757b-4f20-a43f-886db4fdbba2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:42:53.795610 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.795596    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/072d592f-757b-4f20-a43f-886db4fdbba2-kube-api-access-8mdm6" (OuterVolumeSpecName: "kube-api-access-8mdm6") pod "072d592f-757b-4f20-a43f-886db4fdbba2" (UID: "072d592f-757b-4f20-a43f-886db4fdbba2"). InnerVolumeSpecName "kube-api-access-8mdm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:42:53.893640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.893611    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mdm6\" (UniqueName: \"kubernetes.io/projected/072d592f-757b-4f20-a43f-886db4fdbba2-kube-api-access-8mdm6\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:42:53.893640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.893637    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-model-cache\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:42:53.893640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.893649    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/072d592f-757b-4f20-a43f-886db4fdbba2-dshm\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:42:53.893997 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.893660    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/072d592f-757b-4f20-a43f-886db4fdbba2-tls-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:42:53.952283 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.952255    2577 generic.go:356] "Generic (PLEG): container finished" podID="072d592f-757b-4f20-a43f-886db4fdbba2" containerID="b2b6b0a6d6cdaa1e42db3ffd4a99e13d6398fb6035800fda722e375fdb7b5c14" exitCode=0
Sep 07 10:42:53.952437 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.952297    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"
Sep 07 10:42:53.952437 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.952310    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn" event={"ID":"072d592f-757b-4f20-a43f-886db4fdbba2","Type":"ContainerDied","Data":"b2b6b0a6d6cdaa1e42db3ffd4a99e13d6398fb6035800fda722e375fdb7b5c14"}
Sep 07 10:42:53.952437 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.952347    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn" event={"ID":"072d592f-757b-4f20-a43f-886db4fdbba2","Type":"ContainerDied","Data":"3b4af272dc15730cfd14dcbf81769ff2c298d472307932b6fc1351dddb61daf1"}
Sep 07 10:42:53.952437 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.952364    2577 scope.go:117] "RemoveContainer" containerID="b2b6b0a6d6cdaa1e42db3ffd4a99e13d6398fb6035800fda722e375fdb7b5c14"
Sep 07 10:42:53.962182 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.962162    2577 scope.go:117] "RemoveContainer" containerID="b2b6b0a6d6cdaa1e42db3ffd4a99e13d6398fb6035800fda722e375fdb7b5c14"
Sep 07 10:42:53.962490 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:42:53.962472    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2b6b0a6d6cdaa1e42db3ffd4a99e13d6398fb6035800fda722e375fdb7b5c14\": container with ID starting with b2b6b0a6d6cdaa1e42db3ffd4a99e13d6398fb6035800fda722e375fdb7b5c14 not found: ID does not exist" containerID="b2b6b0a6d6cdaa1e42db3ffd4a99e13d6398fb6035800fda722e375fdb7b5c14"
Sep 07 10:42:53.962566 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.962500    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2b6b0a6d6cdaa1e42db3ffd4a99e13d6398fb6035800fda722e375fdb7b5c14"} err="failed to get container status \"b2b6b0a6d6cdaa1e42db3ffd4a99e13d6398fb6035800fda722e375fdb7b5c14\": rpc error: code = NotFound desc = could not find container \"b2b6b0a6d6cdaa1e42db3ffd4a99e13d6398fb6035800fda722e375fdb7b5c14\": container with ID starting with b2b6b0a6d6cdaa1e42db3ffd4a99e13d6398fb6035800fda722e375fdb7b5c14 not found: ID does not exist"
Sep 07 10:42:53.974947 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.974917    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"]
Sep 07 10:42:53.978903 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:53.978873    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-b4061f/test-model-b4061f-kserve-687465d747-rz4wn"]
Sep 07 10:42:54.785435 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:54.785402    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="072d592f-757b-4f20-a43f-886db4fdbba2" path="/var/lib/kubelet/pods/072d592f-757b-4f20-a43f-886db4fdbba2/volumes"
Sep 07 10:42:55.669363 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:55.669323    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:55.731544 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:55.731506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:56.043638 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:56.043605    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:56.583609 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:56.583575    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:58.135787 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:58.135752    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:42:58.169595 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:58.169562    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:58.268985 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:58.268947    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:58.441787 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:58.441704    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:58.785020 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:58.784980    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:42:59.148916 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:42:59.148813    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:01.615389 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:01.615350    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:01.706409 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:01.706365    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:02.736604 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.736570    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"]
Sep 07 10:43:02.737088 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.736965    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca03acb1-c581-4098-87e4-5c3c18768909" containerName="maas-api"
Sep 07 10:43:02.737088 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.736976    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca03acb1-c581-4098-87e4-5c3c18768909" containerName="maas-api"
Sep 07 10:43:02.737088 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.736986    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="072d592f-757b-4f20-a43f-886db4fdbba2" containerName="main"
Sep 07 10:43:02.737088 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.736992    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="072d592f-757b-4f20-a43f-886db4fdbba2" containerName="main"
Sep 07 10:43:02.737088 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.737012    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="829d7efb-68ac-4b7f-af51-59d9d763633d" containerName="maas-api"
Sep 07 10:43:02.737088 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.737017    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="829d7efb-68ac-4b7f-af51-59d9d763633d" containerName="maas-api"
Sep 07 10:43:02.737088 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.737083    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="829d7efb-68ac-4b7f-af51-59d9d763633d" containerName="maas-api"
Sep 07 10:43:02.737088 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.737094    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca03acb1-c581-4098-87e4-5c3c18768909" containerName="maas-api"
Sep 07 10:43:02.737553 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.737102    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="072d592f-757b-4f20-a43f-886db4fdbba2" containerName="main"
Sep 07 10:43:02.741720 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.741697    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:02.744456 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.744434    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-482359dc-serving-cert\""
Sep 07 10:43:02.753638 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.753614    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"]
Sep 07 10:43:02.780744 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.780706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:02.880500 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.880460    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw645\" (UniqueName: \"kubernetes.io/projected/57529d25-b969-4906-a7d4-f0f9aa521eeb-kube-api-access-rw645\") pod \"maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:02.880720 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.880524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57529d25-b969-4906-a7d4-f0f9aa521eeb-maas-api-tls\") pod \"maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:02.880720 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.880553    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57529d25-b969-4906-a7d4-f0f9aa521eeb-openshift-service-ca\") pod \"maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:02.880720 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.880646    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57529d25-b969-4906-a7d4-f0f9aa521eeb-metrics-certs\") pod \"maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:02.881907 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.881879    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:02.981162 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.981096    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57529d25-b969-4906-a7d4-f0f9aa521eeb-metrics-certs\") pod \"maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:02.981354 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.981201    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw645\" (UniqueName: \"kubernetes.io/projected/57529d25-b969-4906-a7d4-f0f9aa521eeb-kube-api-access-rw645\") pod \"maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:02.981354 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.981243    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57529d25-b969-4906-a7d4-f0f9aa521eeb-maas-api-tls\") pod \"maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:02.981354 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.981267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57529d25-b969-4906-a7d4-f0f9aa521eeb-openshift-service-ca\") pod \"maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:02.982165 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.982119    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:02.982330 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.982307    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57529d25-b969-4906-a7d4-f0f9aa521eeb-openshift-service-ca\") pod \"maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:02.983988 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.983965    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57529d25-b969-4906-a7d4-f0f9aa521eeb-metrics-certs\") pod \"maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:02.984081 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.984005    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57529d25-b969-4906-a7d4-f0f9aa521eeb-maas-api-tls\") pod \"maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:02.990412 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:02.990369    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw645\" (UniqueName: \"kubernetes.io/projected/57529d25-b969-4906-a7d4-f0f9aa521eeb-kube-api-access-rw645\") pod \"maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:03.054243 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:03.054210    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:03.293264 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:03.293183    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:03.333978 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:03.333948    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"]
Sep 07 10:43:03.336220 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:43:03.336140    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57529d25_b969_4906_a7d4_f0f9aa521eeb.slice/crio-37d8bdf191b81c0c0d373f61f176789747f82afb6719b049ce7de4c3c722c042 WatchSource:0}: Error finding container 37d8bdf191b81c0c0d373f61f176789747f82afb6719b049ce7de4c3c722c042: Status 404 returned error can't find the container with id 37d8bdf191b81c0c0d373f61f176789747f82afb6719b049ce7de4c3c722c042
Sep 07 10:43:03.392086 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:03.392052    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:04.026028 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:04.025983    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54" event={"ID":"57529d25-b969-4906-a7d4-f0f9aa521eeb","Type":"ContainerStarted","Data":"dd11421bcfb95af5b16716a89a2821d53950c5a030cf9eca2486e1e38021b367"}
Sep 07 10:43:04.026517 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:04.026048    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54" event={"ID":"57529d25-b969-4906-a7d4-f0f9aa521eeb","Type":"ContainerStarted","Data":"37d8bdf191b81c0c0d373f61f176789747f82afb6719b049ce7de4c3c722c042"}
Sep 07 10:43:04.026517 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:04.026068    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:04.044737 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:04.044690    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54" podStartSLOduration=1.454952209 podStartE2EDuration="2.044670706s" totalImagesPullingTime="589.718497ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:43:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:43:04.044155824 +0000 UTC m=+1387.778330066" watchObservedRunningTime="2026-09-07 10:43:04.044670706 +0000 UTC m=+1387.778844941"
Sep 07 10:43:06.544203 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:06.544163    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:43:06.708068 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:06.708039    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:07.412435 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.412399    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"]
Sep 07 10:43:07.412665 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.412641    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7" podUID="c8515b0d-0895-480e-825f-c1ddde9866d1" containerName="maas-api" containerID="cri-o://c7fa61887d2800dffca3299440be2ce3cd86bd2d028303be3863c1c6e46a8380" gracePeriod=30
Sep 07 10:43:07.442050 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.442015    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:07.531457 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.531418    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7" podUID="c8515b0d-0895-480e-825f-c1ddde9866d1" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.65:8443/health\": dial tcp 10.132.0.65:8443: connect: connection refused"
Sep 07 10:43:07.542849 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.542811    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:07.645355 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.645323    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:07.666416 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.666357    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:43:07.725733 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.725699    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8515b0d-0895-480e-825f-c1ddde9866d1-metrics-certs\") pod \"c8515b0d-0895-480e-825f-c1ddde9866d1\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") "
Sep 07 10:43:07.725939 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.725746    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8515b0d-0895-480e-825f-c1ddde9866d1-maas-api-tls\") pod \"c8515b0d-0895-480e-825f-c1ddde9866d1\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") "
Sep 07 10:43:07.725939 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.725762    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85gq4\" (UniqueName: \"kubernetes.io/projected/c8515b0d-0895-480e-825f-c1ddde9866d1-kube-api-access-85gq4\") pod \"c8515b0d-0895-480e-825f-c1ddde9866d1\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") "
Sep 07 10:43:07.725939 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.725905    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8515b0d-0895-480e-825f-c1ddde9866d1-openshift-service-ca\") pod \"c8515b0d-0895-480e-825f-c1ddde9866d1\" (UID: \"c8515b0d-0895-480e-825f-c1ddde9866d1\") "
Sep 07 10:43:07.726278 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.726251    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8515b0d-0895-480e-825f-c1ddde9866d1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c8515b0d-0895-480e-825f-c1ddde9866d1" (UID: "c8515b0d-0895-480e-825f-c1ddde9866d1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:43:07.728120 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.728095    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8515b0d-0895-480e-825f-c1ddde9866d1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c8515b0d-0895-480e-825f-c1ddde9866d1" (UID: "c8515b0d-0895-480e-825f-c1ddde9866d1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:43:07.728120 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.728099    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8515b0d-0895-480e-825f-c1ddde9866d1-kube-api-access-85gq4" (OuterVolumeSpecName: "kube-api-access-85gq4") pod "c8515b0d-0895-480e-825f-c1ddde9866d1" (UID: "c8515b0d-0895-480e-825f-c1ddde9866d1"). InnerVolumeSpecName "kube-api-access-85gq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:43:07.728265 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.728116    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8515b0d-0895-480e-825f-c1ddde9866d1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c8515b0d-0895-480e-825f-c1ddde9866d1" (UID: "c8515b0d-0895-480e-825f-c1ddde9866d1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:43:07.743574 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.743546    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:07.827193 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.827118    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8515b0d-0895-480e-825f-c1ddde9866d1-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:43:07.827193 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.827187    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8515b0d-0895-480e-825f-c1ddde9866d1-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:43:07.827193 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.827197    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8515b0d-0895-480e-825f-c1ddde9866d1-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:43:07.827445 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:07.827206    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85gq4\" (UniqueName: \"kubernetes.io/projected/c8515b0d-0895-480e-825f-c1ddde9866d1-kube-api-access-85gq4\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:43:08.051182 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:08.051145    2577 generic.go:356] "Generic (PLEG): container finished" podID="c8515b0d-0895-480e-825f-c1ddde9866d1" containerID="c7fa61887d2800dffca3299440be2ce3cd86bd2d028303be3863c1c6e46a8380" exitCode=0
Sep 07 10:43:08.051372 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:08.051193    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"
Sep 07 10:43:08.051372 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:08.051240    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7" event={"ID":"c8515b0d-0895-480e-825f-c1ddde9866d1","Type":"ContainerDied","Data":"c7fa61887d2800dffca3299440be2ce3cd86bd2d028303be3863c1c6e46a8380"}
Sep 07 10:43:08.051372 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:08.051295    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7" event={"ID":"c8515b0d-0895-480e-825f-c1ddde9866d1","Type":"ContainerDied","Data":"8ff4afdd94caf0c2e52b0dc43276776ae65aa61ec7f9eadd8db6860e105ff251"}
Sep 07 10:43:08.051372 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:08.051317    2577 scope.go:117] "RemoveContainer" containerID="c7fa61887d2800dffca3299440be2ce3cd86bd2d028303be3863c1c6e46a8380"
Sep 07 10:43:08.061036 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:08.061015    2577 scope.go:117] "RemoveContainer" containerID="c7fa61887d2800dffca3299440be2ce3cd86bd2d028303be3863c1c6e46a8380"
Sep 07 10:43:08.061344 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:43:08.061326    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7fa61887d2800dffca3299440be2ce3cd86bd2d028303be3863c1c6e46a8380\": container with ID starting with c7fa61887d2800dffca3299440be2ce3cd86bd2d028303be3863c1c6e46a8380 not found: ID does not exist" containerID="c7fa61887d2800dffca3299440be2ce3cd86bd2d028303be3863c1c6e46a8380"
Sep 07 10:43:08.061407 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:08.061354    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7fa61887d2800dffca3299440be2ce3cd86bd2d028303be3863c1c6e46a8380"} err="failed to get container status \"c7fa61887d2800dffca3299440be2ce3cd86bd2d028303be3863c1c6e46a8380\": rpc error: code = NotFound desc = could not find container \"c7fa61887d2800dffca3299440be2ce3cd86bd2d028303be3863c1c6e46a8380\": container with ID starting with c7fa61887d2800dffca3299440be2ce3cd86bd2d028303be3863c1c6e46a8380 not found: ID does not exist"
Sep 07 10:43:08.075544 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:08.075511    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"]
Sep 07 10:43:08.079627 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:08.079596    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b4061f-849b888c44-z77j7"]
Sep 07 10:43:08.114504 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:08.114473    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:08.786795 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:08.786763    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8515b0d-0895-480e-825f-c1ddde9866d1" path="/var/lib/kubelet/pods/c8515b0d-0895-480e-825f-c1ddde9866d1/volumes"
Sep 07 10:43:10.038294 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:10.038267    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:13.912083 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:13.912038    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:13.983713 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:13.983680    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:14.207875 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:14.207841    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:14.875325 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:14.875278    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:15.282518 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:15.282486    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:15.380957 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:15.380918    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:16.541927 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:16.541893    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:43:18.705154 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:18.705106    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:18.794581 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:18.794548    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:19.829921 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:19.829888    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"]
Sep 07 10:43:19.830472 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:19.830443    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8515b0d-0895-480e-825f-c1ddde9866d1" containerName="maas-api"
Sep 07 10:43:19.830472 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:19.830468    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8515b0d-0895-480e-825f-c1ddde9866d1" containerName="maas-api"
Sep 07 10:43:19.830685 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:19.830588    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8515b0d-0895-480e-825f-c1ddde9866d1" containerName="maas-api"
Sep 07 10:43:19.835512 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:19.835491    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:19.840169 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:19.840124    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-5a0ff8-serving-cert\""
Sep 07 10:43:19.844344 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:19.844321    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"]
Sep 07 10:43:19.858580 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:19.858548    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:19.935937 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:19.935899    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71510b2c-caf0-49c3-879e-8fdc46cdd211-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:19.936095 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:19.935959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpfhf\" (UniqueName: \"kubernetes.io/projected/71510b2c-caf0-49c3-879e-8fdc46cdd211-kube-api-access-qpfhf\") pod \"maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:19.936095 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:19.936083    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71510b2c-caf0-49c3-879e-8fdc46cdd211-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:19.936272 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:19.936251    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71510b2c-caf0-49c3-879e-8fdc46cdd211-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:19.952501 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:19.952476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:20.037063 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:20.037020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71510b2c-caf0-49c3-879e-8fdc46cdd211-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:20.037063 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:20.037061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpfhf\" (UniqueName: \"kubernetes.io/projected/71510b2c-caf0-49c3-879e-8fdc46cdd211-kube-api-access-qpfhf\") pod \"maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:20.037375 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:20.037186    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71510b2c-caf0-49c3-879e-8fdc46cdd211-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:20.037375 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:20.037255    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71510b2c-caf0-49c3-879e-8fdc46cdd211-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:20.037375 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:43:20.037370    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-serving-cert: secret "maas-api-e2e-shared-a-w3-5a0ff8-serving-cert" not found
Sep 07 10:43:20.037533 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:43:20.037423    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71510b2c-caf0-49c3-879e-8fdc46cdd211-maas-api-tls podName:71510b2c-caf0-49c3-879e-8fdc46cdd211 nodeName:}" failed. No retries permitted until 2026-09-07 10:43:20.53740672 +0000 UTC m=+1404.271580949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/71510b2c-caf0-49c3-879e-8fdc46cdd211-maas-api-tls") pod "maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz" (UID: "71510b2c-caf0-49c3-879e-8fdc46cdd211") : secret "maas-api-e2e-shared-a-w3-5a0ff8-serving-cert" not found
Sep 07 10:43:20.037840 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:20.037820    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71510b2c-caf0-49c3-879e-8fdc46cdd211-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:20.039787 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:20.039765    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71510b2c-caf0-49c3-879e-8fdc46cdd211-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:20.046152 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:20.046107    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpfhf\" (UniqueName: \"kubernetes.io/projected/71510b2c-caf0-49c3-879e-8fdc46cdd211-kube-api-access-qpfhf\") pod \"maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:20.122679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:20.122578    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:20.496998 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:20.496957    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:20.542745 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:20.542714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71510b2c-caf0-49c3-879e-8fdc46cdd211-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:20.545196 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:20.545173    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71510b2c-caf0-49c3-879e-8fdc46cdd211-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:20.592370 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:20.592317    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:20.665607 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:20.665569    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:20.748924 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:20.748847    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:21.031687 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:21.031658    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"]
Sep 07 10:43:21.033587 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:43:21.033562    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71510b2c_caf0_49c3_879e_8fdc46cdd211.slice/crio-59c5916eb994f5e9358aa5c88f943d492dc45205f0f8b832403bb55fa7137caa WatchSource:0}: Error finding container 59c5916eb994f5e9358aa5c88f943d492dc45205f0f8b832403bb55fa7137caa: Status 404 returned error can't find the container with id 59c5916eb994f5e9358aa5c88f943d492dc45205f0f8b832403bb55fa7137caa
Sep 07 10:43:21.135957 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:21.135921    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz" event={"ID":"71510b2c-caf0-49c3-879e-8fdc46cdd211","Type":"ContainerStarted","Data":"59c5916eb994f5e9358aa5c88f943d492dc45205f0f8b832403bb55fa7137caa"}
Sep 07 10:43:22.143206 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:22.143164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz" event={"ID":"71510b2c-caf0-49c3-879e-8fdc46cdd211","Type":"ContainerStarted","Data":"11d2e81292aa16cfe7e63cd10d518143e747f140b05cbe5a4775aa781b9a7b6d"}
Sep 07 10:43:22.143649 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:22.143235    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:22.163239 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:22.163192    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz" podStartSLOduration=2.7476629409999997 podStartE2EDuration="3.163175627s" totalImagesPullingTime="415.512686ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:43:22.161484032 +0000 UTC m=+1405.895658264" watchObservedRunningTime="2026-09-07 10:43:22.163175627 +0000 UTC m=+1405.897349871"
Sep 07 10:43:24.690064 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:24.690011    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:26.674875 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:26.674837    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:27.420462 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:27.420431    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:27.519920 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:27.519879    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:27.888836 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:27.888804    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:28.040371 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:28.040343    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:43:28.153367 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:28.153292    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:43:33.809089 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:33.809017    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:33.908669 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:33.908635    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:40.915365 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:40.915327    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:43.797773 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:43.797736    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:44.115791 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:44.115698    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:44.485726 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:44.485675    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:44.578283 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:44.578240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:44.748701 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:44.748600    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:45.093738 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.093643    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"]
Sep 07 10:43:45.094306 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.093896    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54" podUID="57529d25-b969-4906-a7d4-f0f9aa521eeb" containerName="maas-api" containerID="cri-o://dd11421bcfb95af5b16716a89a2821d53950c5a030cf9eca2486e1e38021b367" gracePeriod=30
Sep 07 10:43:45.124624 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.124595    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:45.200303 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.200266    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:45.266785 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.266749    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:45.285856 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.285825    2577 generic.go:356] "Generic (PLEG): container finished" podID="57529d25-b969-4906-a7d4-f0f9aa521eeb" containerID="dd11421bcfb95af5b16716a89a2821d53950c5a030cf9eca2486e1e38021b367" exitCode=0
Sep 07 10:43:45.286036 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.285903    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54" event={"ID":"57529d25-b969-4906-a7d4-f0f9aa521eeb","Type":"ContainerDied","Data":"dd11421bcfb95af5b16716a89a2821d53950c5a030cf9eca2486e1e38021b367"}
Sep 07 10:43:45.337774 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.337739    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:45.353468 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.353407    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:45.405508 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.405458    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:45.472713 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.471523    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57529d25-b969-4906-a7d4-f0f9aa521eeb-openshift-service-ca\") pod \"57529d25-b969-4906-a7d4-f0f9aa521eeb\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") "
Sep 07 10:43:45.472713 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.471571    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57529d25-b969-4906-a7d4-f0f9aa521eeb-maas-api-tls\") pod \"57529d25-b969-4906-a7d4-f0f9aa521eeb\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") "
Sep 07 10:43:45.472713 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.471641    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rw645\" (UniqueName: \"kubernetes.io/projected/57529d25-b969-4906-a7d4-f0f9aa521eeb-kube-api-access-rw645\") pod \"57529d25-b969-4906-a7d4-f0f9aa521eeb\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") "
Sep 07 10:43:45.472713 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.471734    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57529d25-b969-4906-a7d4-f0f9aa521eeb-metrics-certs\") pod \"57529d25-b969-4906-a7d4-f0f9aa521eeb\" (UID: \"57529d25-b969-4906-a7d4-f0f9aa521eeb\") "
Sep 07 10:43:45.473288 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.472805    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57529d25-b969-4906-a7d4-f0f9aa521eeb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "57529d25-b969-4906-a7d4-f0f9aa521eeb" (UID: "57529d25-b969-4906-a7d4-f0f9aa521eeb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:43:45.476195 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.476164    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57529d25-b969-4906-a7d4-f0f9aa521eeb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "57529d25-b969-4906-a7d4-f0f9aa521eeb" (UID: "57529d25-b969-4906-a7d4-f0f9aa521eeb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:43:45.476446 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.476424    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57529d25-b969-4906-a7d4-f0f9aa521eeb-kube-api-access-rw645" (OuterVolumeSpecName: "kube-api-access-rw645") pod "57529d25-b969-4906-a7d4-f0f9aa521eeb" (UID: "57529d25-b969-4906-a7d4-f0f9aa521eeb"). InnerVolumeSpecName "kube-api-access-rw645". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:43:45.476704 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.476686    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57529d25-b969-4906-a7d4-f0f9aa521eeb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "57529d25-b969-4906-a7d4-f0f9aa521eeb" (UID: "57529d25-b969-4906-a7d4-f0f9aa521eeb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:43:45.572693 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.572645    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57529d25-b969-4906-a7d4-f0f9aa521eeb-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:43:45.572693 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.572684    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57529d25-b969-4906-a7d4-f0f9aa521eeb-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:43:45.572693 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.572699    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw645\" (UniqueName: \"kubernetes.io/projected/57529d25-b969-4906-a7d4-f0f9aa521eeb-kube-api-access-rw645\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:43:45.573046 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.572713    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57529d25-b969-4906-a7d4-f0f9aa521eeb-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:43:45.710716 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:45.710687    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:46.294352 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:46.294319    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"
Sep 07 10:43:46.294352 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:46.294340    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54" event={"ID":"57529d25-b969-4906-a7d4-f0f9aa521eeb","Type":"ContainerDied","Data":"37d8bdf191b81c0c0d373f61f176789747f82afb6719b049ce7de4c3c722c042"}
Sep 07 10:43:46.294805 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:46.294406    2577 scope.go:117] "RemoveContainer" containerID="dd11421bcfb95af5b16716a89a2821d53950c5a030cf9eca2486e1e38021b367"
Sep 07 10:43:46.316657 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:46.316627    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"]
Sep 07 10:43:46.319052 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:46.319029    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-482359dc-7dbcbf9f44-zcg54"]
Sep 07 10:43:46.519637 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:46.519607    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:46.786291 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:46.786247    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57529d25-b969-4906-a7d4-f0f9aa521eeb" path="/var/lib/kubelet/pods/57529d25-b969-4906-a7d4-f0f9aa521eeb/volumes"
Sep 07 10:43:47.816535 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:47.816447    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:47.915476 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:47.915439    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:48.957413 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:48.957379    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"]
Sep 07 10:43:48.957996 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:48.957979    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57529d25-b969-4906-a7d4-f0f9aa521eeb" containerName="maas-api"
Sep 07 10:43:48.958041 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:48.958000    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="57529d25-b969-4906-a7d4-f0f9aa521eeb" containerName="maas-api"
Sep 07 10:43:48.958156 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:48.958144    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="57529d25-b969-4906-a7d4-f0f9aa521eeb" containerName="maas-api"
Sep 07 10:43:48.962946 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:48.962919    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:48.966992 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:48.966969    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-659410-serving-cert\""
Sep 07 10:43:48.982431 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:48.982402    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"]
Sep 07 10:43:49.005516 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.005477    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55b54502-7ac4-474c-ac1e-1ace55a068f4-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:49.005686 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.005532    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55b54502-7ac4-474c-ac1e-1ace55a068f4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:49.005686 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.005567    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55b54502-7ac4-474c-ac1e-1ace55a068f4-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:49.005686 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.005585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsvqr\" (UniqueName: \"kubernetes.io/projected/55b54502-7ac4-474c-ac1e-1ace55a068f4-kube-api-access-tsvqr\") pod \"maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:49.043548 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.043516    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:49.106706 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.106674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55b54502-7ac4-474c-ac1e-1ace55a068f4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:49.106706 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.106710    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55b54502-7ac4-474c-ac1e-1ace55a068f4-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:49.106972 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.106740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsvqr\" (UniqueName: \"kubernetes.io/projected/55b54502-7ac4-474c-ac1e-1ace55a068f4-kube-api-access-tsvqr\") pod \"maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:49.106972 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.106854    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55b54502-7ac4-474c-ac1e-1ace55a068f4-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:49.107542 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.107467    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55b54502-7ac4-474c-ac1e-1ace55a068f4-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:49.109493 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.109473    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55b54502-7ac4-474c-ac1e-1ace55a068f4-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:49.109572 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.109508    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55b54502-7ac4-474c-ac1e-1ace55a068f4-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:49.124640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.124616    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsvqr\" (UniqueName: \"kubernetes.io/projected/55b54502-7ac4-474c-ac1e-1ace55a068f4-kube-api-access-tsvqr\") pod \"maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:49.160566 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.160524    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:49.260988 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.260895    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:49.275602 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.275576    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:49.539792 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.539703    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:43:49.558573 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.558551    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"]
Sep 07 10:43:49.561094 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:43:49.561053    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55b54502_7ac4_474c_ac1e_1ace55a068f4.slice/crio-681f1f58230b7a1e9e547a7966329834c3a422bb46c256c314e6dcac85100b47 WatchSource:0}: Error finding container 681f1f58230b7a1e9e547a7966329834c3a422bb46c256c314e6dcac85100b47: Status 404 returned error can't find the container with id 681f1f58230b7a1e9e547a7966329834c3a422bb46c256c314e6dcac85100b47
Sep 07 10:43:49.578368 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.578336    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:49.691317 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:49.691280    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:50.323845 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:50.323806    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg" event={"ID":"55b54502-7ac4-474c-ac1e-1ace55a068f4","Type":"ContainerStarted","Data":"ded16470125324f22277e9c24618b59e942d28d5adaf0b82406ee3ce4c8b1266"}
Sep 07 10:43:50.324268 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:50.323868    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg" event={"ID":"55b54502-7ac4-474c-ac1e-1ace55a068f4","Type":"ContainerStarted","Data":"681f1f58230b7a1e9e547a7966329834c3a422bb46c256c314e6dcac85100b47"}
Sep 07 10:43:50.324268 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:50.323931    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:50.341915 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:50.341860    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg" podStartSLOduration=2.004079411 podStartE2EDuration="2.341844476s" totalImagesPullingTime="337.765065ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:43:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:43:50.339848578 +0000 UTC m=+1434.074022810" watchObservedRunningTime="2026-09-07 10:43:50.341844476 +0000 UTC m=+1434.076018768"
Sep 07 10:43:51.683313 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:51.683276    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:51.747904 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:51.747868    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:51.993745 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:51.993703    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:52.354516 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:52.354420    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:52.454409 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:52.454376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:52.611857 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:52.611772    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:56.336368 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:56.336334    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:43:58.539316 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:58.539274    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:58.600551 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:58.600511    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:58.694024 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:58.693986    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:58.781967 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:58.781930    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:43:58.944995 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:43:58.944907    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:00.535160 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:00.530506    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:44:04.677734 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:04.677695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:04.795988 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:04.795942    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:05.683306 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:05.683263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:06.003998 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:06.003970    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:06.068753 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:06.068712    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:08.903685 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:08.903604    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:08.993370 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:08.993335    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:10.692287 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.692245    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"]
Sep 07 10:44:10.697655 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.697630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:10.700186 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.700163    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-a158df-serving-cert\""
Sep 07 10:44:10.708213 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.708188    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"]
Sep 07 10:44:10.730671 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.730637    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:10.797152 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.797110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28dfc33b-4247-463b-88c6-76dfb11a3925-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:10.797345 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.797172    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28dfc33b-4247-463b-88c6-76dfb11a3925-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:10.797345 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.797296    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28dfc33b-4247-463b-88c6-76dfb11a3925-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:10.797345 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.797327    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt6wt\" (UniqueName: \"kubernetes.io/projected/28dfc33b-4247-463b-88c6-76dfb11a3925-kube-api-access-dt6wt\") pod \"maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:10.833071 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.833030    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:10.898452 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.898416    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28dfc33b-4247-463b-88c6-76dfb11a3925-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:10.898648 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.898460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28dfc33b-4247-463b-88c6-76dfb11a3925-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:10.898648 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.898581    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28dfc33b-4247-463b-88c6-76dfb11a3925-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:10.898648 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.898610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt6wt\" (UniqueName: \"kubernetes.io/projected/28dfc33b-4247-463b-88c6-76dfb11a3925-kube-api-access-dt6wt\") pod \"maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:10.899361 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.899271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28dfc33b-4247-463b-88c6-76dfb11a3925-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:10.901114 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.901093    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28dfc33b-4247-463b-88c6-76dfb11a3925-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:10.901281 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.901263    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28dfc33b-4247-463b-88c6-76dfb11a3925-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:10.906912 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.906892    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt6wt\" (UniqueName: \"kubernetes.io/projected/28dfc33b-4247-463b-88c6-76dfb11a3925-kube-api-access-dt6wt\") pod \"maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:10.933221 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:10.933198    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:11.009988 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:11.009957    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:11.296307 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:11.296256    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"]
Sep 07 10:44:11.298179 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:44:11.298095    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28dfc33b_4247_463b_88c6_76dfb11a3925.slice/crio-a9282e8b6839745591011c9762e7bbadb6036f535c738c84f875502f8dc6c221 WatchSource:0}: Error finding container a9282e8b6839745591011c9762e7bbadb6036f535c738c84f875502f8dc6c221: Status 404 returned error can't find the container with id a9282e8b6839745591011c9762e7bbadb6036f535c738c84f875502f8dc6c221
Sep 07 10:44:11.341687 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:11.341657    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:11.441042 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:11.441004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:11.446613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:11.446589    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv" event={"ID":"28dfc33b-4247-463b-88c6-76dfb11a3925","Type":"ContainerStarted","Data":"a9282e8b6839745591011c9762e7bbadb6036f535c738c84f875502f8dc6c221"}
Sep 07 10:44:11.763841 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:11.763797    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:12.453585 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:12.453544    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv" event={"ID":"28dfc33b-4247-463b-88c6-76dfb11a3925","Type":"ContainerStarted","Data":"469a95822f3e0c0a1c2ae306a27dce3424c5006cbab74856a1582b41a7147b67"}
Sep 07 10:44:12.453769 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:12.453616    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:12.466969 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:12.466934    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:12.473808 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:12.473765    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv" podStartSLOduration=2.004878485 podStartE2EDuration="2.473751373s" totalImagesPullingTime="468.872888ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:44:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:44:12.472757146 +0000 UTC m=+1456.206931378" watchObservedRunningTime="2026-09-07 10:44:12.473751373 +0000 UTC m=+1456.207925606"
Sep 07 10:44:12.895936 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:12.895804    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:14.725021 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:14.724985    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:15.428523 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:15.428479    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:44:18.465304 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:18.465271    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:44:18.760622 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:18.760545    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:29.472667 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:29.472630    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:44:34.811465 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:34.811431    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:34.909903 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:34.909870    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:35.009896 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:35.009862    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:38.236578 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:38.236539    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:38.336345 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:38.336312    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:39.422371 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.422294    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"]
Sep 07 10:44:39.426098 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.426081    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:39.428883 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.428862    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-5e934f-serving-cert\""
Sep 07 10:44:39.449529 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.449504    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"]
Sep 07 10:44:39.452831 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.452805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:39.452941 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.452845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgv5r\" (UniqueName: \"kubernetes.io/projected/1aebccc8-116d-474f-a504-6e8d4906635c-kube-api-access-hgv5r\") pod \"maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:39.453000 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.452979    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aebccc8-116d-474f-a504-6e8d4906635c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:39.453077 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.453052    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:39.511354 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.511316    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:39.554528 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.554485    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:39.554718 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.554532    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgv5r\" (UniqueName: \"kubernetes.io/projected/1aebccc8-116d-474f-a504-6e8d4906635c-kube-api-access-hgv5r\") pod \"maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:39.554718 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.554602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aebccc8-116d-474f-a504-6e8d4906635c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:39.554718 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:44:39.554632    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-serving-cert: secret "maas-api-e2e-shared-b-w2-5e934f-serving-cert" not found
Sep 07 10:44:39.554718 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.554650    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:39.554990 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:44:39.554729    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-maas-api-tls podName:1aebccc8-116d-474f-a504-6e8d4906635c nodeName:}" failed. No retries permitted until 2026-09-07 10:44:40.054704734 +0000 UTC m=+1483.788878958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-maas-api-tls") pod "maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn" (UID: "1aebccc8-116d-474f-a504-6e8d4906635c") : secret "maas-api-e2e-shared-b-w2-5e934f-serving-cert" not found
Sep 07 10:44:39.555317 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.555295    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aebccc8-116d-474f-a504-6e8d4906635c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:39.557285 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.557263    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:39.563490 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.563466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgv5r\" (UniqueName: \"kubernetes.io/projected/1aebccc8-116d-474f-a504-6e8d4906635c-kube-api-access-hgv5r\") pod \"maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:39.611721 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.611677    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:39.992236 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:39.992192    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:40.060041 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:40.059994    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:40.060271 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:44:40.060181    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-serving-cert: secret "maas-api-e2e-shared-b-w2-5e934f-serving-cert" not found
Sep 07 10:44:40.060271 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:44:40.060251    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-maas-api-tls podName:1aebccc8-116d-474f-a504-6e8d4906635c nodeName:}" failed. No retries permitted until 2026-09-07 10:44:41.060235682 +0000 UTC m=+1484.794409894 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-maas-api-tls") pod "maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn" (UID: "1aebccc8-116d-474f-a504-6e8d4906635c") : secret "maas-api-e2e-shared-b-w2-5e934f-serving-cert" not found
Sep 07 10:44:40.091661 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:40.091628    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:40.178118 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:40.178085    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:44:41.070220 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:41.070182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:41.073274 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:41.073248    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:41.236969 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:41.236936    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:41.515233 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:41.515197    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"]
Sep 07 10:44:41.516720 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:44:41.516693    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aebccc8_116d_474f_a504_6e8d4906635c.slice/crio-58fc0c903845094bae5c5d9f07c21118609380cd9939b8221e1376cf2ad6cf38 WatchSource:0}: Error finding container 58fc0c903845094bae5c5d9f07c21118609380cd9939b8221e1376cf2ad6cf38: Status 404 returned error can't find the container with id 58fc0c903845094bae5c5d9f07c21118609380cd9939b8221e1376cf2ad6cf38
Sep 07 10:44:41.640578 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:41.640534    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn" event={"ID":"1aebccc8-116d-474f-a504-6e8d4906635c","Type":"ContainerStarted","Data":"58fc0c903845094bae5c5d9f07c21118609380cd9939b8221e1376cf2ad6cf38"}
Sep 07 10:44:42.648149 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:42.648096    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn" event={"ID":"1aebccc8-116d-474f-a504-6e8d4906635c","Type":"ContainerStarted","Data":"cbadc4b3911c191f8f545ad689b00629304452777ad1d8bb2b40eab3ff4dbb51"}
Sep 07 10:44:42.648533 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:42.648179    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:42.668515 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:42.668465    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn" podStartSLOduration=3.144278111 podStartE2EDuration="3.668448721s" totalImagesPullingTime="524.17061ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:44:42.665746915 +0000 UTC m=+1486.399921160" watchObservedRunningTime="2026-09-07 10:44:42.668448721 +0000 UTC m=+1486.402622953"
Sep 07 10:44:47.341678 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:47.341622    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:44:48.660224 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:48.660188    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:44:56.851060 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:56.851031    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:44:56.856319 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:56.856296    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:44:56.943687 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:44:56.943658    2577 scope.go:117] "RemoveContainer" containerID="e32875e39bbd42c402fc2ad57b544c653f50bd9eadf1485fdbf9b9c8508ed49b"
Sep 07 10:45:03.385341 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:03.385297    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:45:04.295243 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:04.295207    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7750-139678922966336"]
Sep 07 10:45:04.300318 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:04.300297    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966336"
Sep 07 10:45:04.302662 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:04.302638    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7750-139678922966336"]
Sep 07 10:45:04.302777 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:04.302706    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-xwqv8\""
Sep 07 10:45:04.396439 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:04.396393    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkzjk\" (UniqueName: \"kubernetes.io/projected/1989b92f-a7ba-4e83-99d1-1cde07f2ada5-kube-api-access-gkzjk\") pod \"test-curl-7750-139678922966336\" (UID: \"1989b92f-a7ba-4e83-99d1-1cde07f2ada5\") " pod="opendatahub/test-curl-7750-139678922966336"
Sep 07 10:45:04.498016 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:04.497981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkzjk\" (UniqueName: \"kubernetes.io/projected/1989b92f-a7ba-4e83-99d1-1cde07f2ada5-kube-api-access-gkzjk\") pod \"test-curl-7750-139678922966336\" (UID: \"1989b92f-a7ba-4e83-99d1-1cde07f2ada5\") " pod="opendatahub/test-curl-7750-139678922966336"
Sep 07 10:45:04.509107 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:04.509068    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkzjk\" (UniqueName: \"kubernetes.io/projected/1989b92f-a7ba-4e83-99d1-1cde07f2ada5-kube-api-access-gkzjk\") pod \"test-curl-7750-139678922966336\" (UID: \"1989b92f-a7ba-4e83-99d1-1cde07f2ada5\") " pod="opendatahub/test-curl-7750-139678922966336"
Sep 07 10:45:04.611061 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:04.610981    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966336"
Sep 07 10:45:04.882065 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:04.882038    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7750-139678922966336"]
Sep 07 10:45:04.883517 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:45:04.883484    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1989b92f_a7ba_4e83_99d1_1cde07f2ada5.slice/crio-c88265495897fbb5e585485c612f0c1cab5857437582b11f0656a027a3a43fc8 WatchSource:0}: Error finding container c88265495897fbb5e585485c612f0c1cab5857437582b11f0656a027a3a43fc8: Status 404 returned error can't find the container with id c88265495897fbb5e585485c612f0c1cab5857437582b11f0656a027a3a43fc8
Sep 07 10:45:05.789408 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:05.789370    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7750-139678922966336" event={"ID":"1989b92f-a7ba-4e83-99d1-1cde07f2ada5","Type":"ContainerStarted","Data":"c88265495897fbb5e585485c612f0c1cab5857437582b11f0656a027a3a43fc8"}
Sep 07 10:45:06.797060 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:06.797027    2577 generic.go:356] "Generic (PLEG): container finished" podID="1989b92f-a7ba-4e83-99d1-1cde07f2ada5" containerID="41307d2f87099009013a4aa076ca2b984ca696f4c6c69903c73dfbd2925969d1" exitCode=0
Sep 07 10:45:06.797485 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:06.797112    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7750-139678922966336" event={"ID":"1989b92f-a7ba-4e83-99d1-1cde07f2ada5","Type":"ContainerDied","Data":"41307d2f87099009013a4aa076ca2b984ca696f4c6c69903c73dfbd2925969d1"}
Sep 07 10:45:07.934291 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:07.934256    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966336"
Sep 07 10:45:07.957646 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:07.957613    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7750-139678922966336_1989b92f-a7ba-4e83-99d1-1cde07f2ada5/test-curl-7750-139678922966336/0.log"
Sep 07 10:45:07.987778 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:07.987743    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7750-139678922966336"]
Sep 07 10:45:07.991596 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:07.991572    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7750-139678922966336"]
Sep 07 10:45:08.033094 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.033059    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkzjk\" (UniqueName: \"kubernetes.io/projected/1989b92f-a7ba-4e83-99d1-1cde07f2ada5-kube-api-access-gkzjk\") pod \"1989b92f-a7ba-4e83-99d1-1cde07f2ada5\" (UID: \"1989b92f-a7ba-4e83-99d1-1cde07f2ada5\") "
Sep 07 10:45:08.035360 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.035329    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1989b92f-a7ba-4e83-99d1-1cde07f2ada5-kube-api-access-gkzjk" (OuterVolumeSpecName: "kube-api-access-gkzjk") pod "1989b92f-a7ba-4e83-99d1-1cde07f2ada5" (UID: "1989b92f-a7ba-4e83-99d1-1cde07f2ada5"). InnerVolumeSpecName "kube-api-access-gkzjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:45:08.134504 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.134417    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkzjk\" (UniqueName: \"kubernetes.io/projected/1989b92f-a7ba-4e83-99d1-1cde07f2ada5-kube-api-access-gkzjk\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:45:08.143746 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.143720    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7750-139678922966336"]
Sep 07 10:45:08.144172 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.144159    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1989b92f-a7ba-4e83-99d1-1cde07f2ada5" containerName="test-curl-7750-139678922966336"
Sep 07 10:45:08.144218 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.144174    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1989b92f-a7ba-4e83-99d1-1cde07f2ada5" containerName="test-curl-7750-139678922966336"
Sep 07 10:45:08.144253 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.144245    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1989b92f-a7ba-4e83-99d1-1cde07f2ada5" containerName="test-curl-7750-139678922966336"
Sep 07 10:45:08.147353 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.147336    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966336"
Sep 07 10:45:08.154206 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.154183    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7750-139678922966336"]
Sep 07 10:45:08.337037 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.336992    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94z72\" (UniqueName: \"kubernetes.io/projected/be679bb0-2172-407c-9090-83daa3eb77a6-kube-api-access-94z72\") pod \"test-curl-7750-139678922966336\" (UID: \"be679bb0-2172-407c-9090-83daa3eb77a6\") " pod="opendatahub/test-curl-7750-139678922966336"
Sep 07 10:45:08.437950 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.437860    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94z72\" (UniqueName: \"kubernetes.io/projected/be679bb0-2172-407c-9090-83daa3eb77a6-kube-api-access-94z72\") pod \"test-curl-7750-139678922966336\" (UID: \"be679bb0-2172-407c-9090-83daa3eb77a6\") " pod="opendatahub/test-curl-7750-139678922966336"
Sep 07 10:45:08.446211 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.446179    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94z72\" (UniqueName: \"kubernetes.io/projected/be679bb0-2172-407c-9090-83daa3eb77a6-kube-api-access-94z72\") pod \"test-curl-7750-139678922966336\" (UID: \"be679bb0-2172-407c-9090-83daa3eb77a6\") " pod="opendatahub/test-curl-7750-139678922966336"
Sep 07 10:45:08.458096 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.458060    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966336"
Sep 07 10:45:08.733261 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.733179    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7750-139678922966336"]
Sep 07 10:45:08.736660 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:45:08.736627    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe679bb0_2172_407c_9090_83daa3eb77a6.slice/crio-d2d7792b388fe60c9a2dd3fb0c494af0e14e47c70e2f7b078251b51a5e145d25 WatchSource:0}: Error finding container d2d7792b388fe60c9a2dd3fb0c494af0e14e47c70e2f7b078251b51a5e145d25: Status 404 returned error can't find the container with id d2d7792b388fe60c9a2dd3fb0c494af0e14e47c70e2f7b078251b51a5e145d25
Sep 07 10:45:08.786956 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.786927    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1989b92f-a7ba-4e83-99d1-1cde07f2ada5" path="/var/lib/kubelet/pods/1989b92f-a7ba-4e83-99d1-1cde07f2ada5/volumes"
Sep 07 10:45:08.809799 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.809768    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7750-139678922966336" event={"ID":"be679bb0-2172-407c-9090-83daa3eb77a6","Type":"ContainerStarted","Data":"d2d7792b388fe60c9a2dd3fb0c494af0e14e47c70e2f7b078251b51a5e145d25"}
Sep 07 10:45:08.811076 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.811050    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966336"
Sep 07 10:45:08.811209 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:08.811077    2577 scope.go:117] "RemoveContainer" containerID="41307d2f87099009013a4aa076ca2b984ca696f4c6c69903c73dfbd2925969d1"
Sep 07 10:45:09.819384 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:09.819294    2577 generic.go:356] "Generic (PLEG): container finished" podID="be679bb0-2172-407c-9090-83daa3eb77a6" containerID="45da76da28aec36b48bb5cee548e359d30e1133736f50d36a568d83246c1a2aa" exitCode=0
Sep 07 10:45:09.819384 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:09.819356    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7750-139678922966336" event={"ID":"be679bb0-2172-407c-9090-83daa3eb77a6","Type":"ContainerDied","Data":"45da76da28aec36b48bb5cee548e359d30e1133736f50d36a568d83246c1a2aa"}
Sep 07 10:45:10.970761 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:10.970732    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966336"
Sep 07 10:45:10.994756 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:10.994715    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7750-139678922966336_be679bb0-2172-407c-9090-83daa3eb77a6/test-curl-7750-139678922966336/0.log"
Sep 07 10:45:11.028441 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.028189    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7750-139678922966336"]
Sep 07 10:45:11.030993 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.030968    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7750-139678922966336"]
Sep 07 10:45:11.161337 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.161229    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94z72\" (UniqueName: \"kubernetes.io/projected/be679bb0-2172-407c-9090-83daa3eb77a6-kube-api-access-94z72\") pod \"be679bb0-2172-407c-9090-83daa3eb77a6\" (UID: \"be679bb0-2172-407c-9090-83daa3eb77a6\") "
Sep 07 10:45:11.163542 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.163509    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be679bb0-2172-407c-9090-83daa3eb77a6-kube-api-access-94z72" (OuterVolumeSpecName: "kube-api-access-94z72") pod "be679bb0-2172-407c-9090-83daa3eb77a6" (UID: "be679bb0-2172-407c-9090-83daa3eb77a6"). InnerVolumeSpecName "kube-api-access-94z72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:45:11.186176 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.186125    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7750-139678922966912"]
Sep 07 10:45:11.186569 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.186555    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be679bb0-2172-407c-9090-83daa3eb77a6" containerName="test-curl-7750-139678922966336"
Sep 07 10:45:11.186609 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.186571    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="be679bb0-2172-407c-9090-83daa3eb77a6" containerName="test-curl-7750-139678922966336"
Sep 07 10:45:11.186647 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.186638    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="be679bb0-2172-407c-9090-83daa3eb77a6" containerName="test-curl-7750-139678922966336"
Sep 07 10:45:11.189701 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.189683    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966912"
Sep 07 10:45:11.193818 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.193790    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7750-139678922966912"]
Sep 07 10:45:11.262877 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.262831    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94z72\" (UniqueName: \"kubernetes.io/projected/be679bb0-2172-407c-9090-83daa3eb77a6-kube-api-access-94z72\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:45:11.363791 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.363731    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g2h8\" (UniqueName: \"kubernetes.io/projected/f4118145-8f14-4837-adff-693b006dc37a-kube-api-access-8g2h8\") pod \"test-curl-7750-139678922966912\" (UID: \"f4118145-8f14-4837-adff-693b006dc37a\") " pod="opendatahub/test-curl-7750-139678922966912"
Sep 07 10:45:11.464299 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.464255    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g2h8\" (UniqueName: \"kubernetes.io/projected/f4118145-8f14-4837-adff-693b006dc37a-kube-api-access-8g2h8\") pod \"test-curl-7750-139678922966912\" (UID: \"f4118145-8f14-4837-adff-693b006dc37a\") " pod="opendatahub/test-curl-7750-139678922966912"
Sep 07 10:45:11.473045 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.473003    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g2h8\" (UniqueName: \"kubernetes.io/projected/f4118145-8f14-4837-adff-693b006dc37a-kube-api-access-8g2h8\") pod \"test-curl-7750-139678922966912\" (UID: \"f4118145-8f14-4837-adff-693b006dc37a\") " pod="opendatahub/test-curl-7750-139678922966912"
Sep 07 10:45:11.500824 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.500791    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966912"
Sep 07 10:45:11.774958 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.774925    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7750-139678922966912"]
Sep 07 10:45:11.776948 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:45:11.776880    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4118145_8f14_4837_adff_693b006dc37a.slice/crio-a7817ca564770f9d4f78a597e23744cfe38d2b585107278626f93369e0d6e75b WatchSource:0}: Error finding container a7817ca564770f9d4f78a597e23744cfe38d2b585107278626f93369e0d6e75b: Status 404 returned error can't find the container with id a7817ca564770f9d4f78a597e23744cfe38d2b585107278626f93369e0d6e75b
Sep 07 10:45:11.833650 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.833613    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966336"
Sep 07 10:45:11.833650 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.833636    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2d7792b388fe60c9a2dd3fb0c494af0e14e47c70e2f7b078251b51a5e145d25"
Sep 07 10:45:11.834937 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:11.834903    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7750-139678922966912" event={"ID":"f4118145-8f14-4837-adff-693b006dc37a","Type":"ContainerStarted","Data":"a7817ca564770f9d4f78a597e23744cfe38d2b585107278626f93369e0d6e75b"}
Sep 07 10:45:11.844066 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:45:11.844039    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7750-139678922966336\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="be679bb0-2172-407c-9090-83daa3eb77a6" pod="opendatahub/test-curl-7750-139678922966336"
Sep 07 10:45:12.789911 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:12.789878    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be679bb0-2172-407c-9090-83daa3eb77a6" path="/var/lib/kubelet/pods/be679bb0-2172-407c-9090-83daa3eb77a6/volumes"
Sep 07 10:45:12.841948 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:12.841864    2577 generic.go:356] "Generic (PLEG): container finished" podID="f4118145-8f14-4837-adff-693b006dc37a" containerID="fc454af22186bd310df6ec032f8499709033569e522b0f1fbab9cd29b2d82416" exitCode=0
Sep 07 10:45:12.842097 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:12.841952    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7750-139678922966912" event={"ID":"f4118145-8f14-4837-adff-693b006dc37a","Type":"ContainerDied","Data":"fc454af22186bd310df6ec032f8499709033569e522b0f1fbab9cd29b2d82416"}
Sep 07 10:45:13.984189 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:13.984162    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966912"
Sep 07 10:45:14.004291 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.004252    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7750-139678922966912_f4118145-8f14-4837-adff-693b006dc37a/test-curl-7750-139678922966912/0.log"
Sep 07 10:45:14.026429 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.026395    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7750-139678922966912"]
Sep 07 10:45:14.031038 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.031009    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7750-139678922966912"]
Sep 07 10:45:14.087968 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.087939    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8g2h8\" (UniqueName: \"kubernetes.io/projected/f4118145-8f14-4837-adff-693b006dc37a-kube-api-access-8g2h8\") pod \"f4118145-8f14-4837-adff-693b006dc37a\" (UID: \"f4118145-8f14-4837-adff-693b006dc37a\") "
Sep 07 10:45:14.090216 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.090169    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4118145-8f14-4837-adff-693b006dc37a-kube-api-access-8g2h8" (OuterVolumeSpecName: "kube-api-access-8g2h8") pod "f4118145-8f14-4837-adff-693b006dc37a" (UID: "f4118145-8f14-4837-adff-693b006dc37a"). InnerVolumeSpecName "kube-api-access-8g2h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:45:14.182258 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.182149    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7750-139678922966912"]
Sep 07 10:45:14.182596 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.182582    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4118145-8f14-4837-adff-693b006dc37a" containerName="test-curl-7750-139678922966912"
Sep 07 10:45:14.182642 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.182598    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4118145-8f14-4837-adff-693b006dc37a" containerName="test-curl-7750-139678922966912"
Sep 07 10:45:14.182688 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.182679    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4118145-8f14-4837-adff-693b006dc37a" containerName="test-curl-7750-139678922966912"
Sep 07 10:45:14.185717 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.185702    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966912"
Sep 07 10:45:14.188902 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.188872    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8g2h8\" (UniqueName: \"kubernetes.io/projected/f4118145-8f14-4837-adff-693b006dc37a-kube-api-access-8g2h8\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:45:14.192249 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.192226    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7750-139678922966912"]
Sep 07 10:45:14.290309 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.290272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd5w6\" (UniqueName: \"kubernetes.io/projected/5f5aa6f2-3a55-459c-8a2d-997d874c3ae6-kube-api-access-gd5w6\") pod \"test-curl-7750-139678922966912\" (UID: \"5f5aa6f2-3a55-459c-8a2d-997d874c3ae6\") " pod="opendatahub/test-curl-7750-139678922966912"
Sep 07 10:45:14.391308 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.391275    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd5w6\" (UniqueName: \"kubernetes.io/projected/5f5aa6f2-3a55-459c-8a2d-997d874c3ae6-kube-api-access-gd5w6\") pod \"test-curl-7750-139678922966912\" (UID: \"5f5aa6f2-3a55-459c-8a2d-997d874c3ae6\") " pod="opendatahub/test-curl-7750-139678922966912"
Sep 07 10:45:14.399491 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.399461    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd5w6\" (UniqueName: \"kubernetes.io/projected/5f5aa6f2-3a55-459c-8a2d-997d874c3ae6-kube-api-access-gd5w6\") pod \"test-curl-7750-139678922966912\" (UID: \"5f5aa6f2-3a55-459c-8a2d-997d874c3ae6\") " pod="opendatahub/test-curl-7750-139678922966912"
Sep 07 10:45:14.497004 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.496970    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966912"
Sep 07 10:45:14.770119 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.770096    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7750-139678922966912"]
Sep 07 10:45:14.771544 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:45:14.771516    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f5aa6f2_3a55_459c_8a2d_997d874c3ae6.slice/crio-cb26e53d88e46a1bcb76f37a2e52710a920e181c56f7abc9e67d2b07471807df WatchSource:0}: Error finding container cb26e53d88e46a1bcb76f37a2e52710a920e181c56f7abc9e67d2b07471807df: Status 404 returned error can't find the container with id cb26e53d88e46a1bcb76f37a2e52710a920e181c56f7abc9e67d2b07471807df
Sep 07 10:45:14.785463 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.785439    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4118145-8f14-4837-adff-693b006dc37a" path="/var/lib/kubelet/pods/f4118145-8f14-4837-adff-693b006dc37a/volumes"
Sep 07 10:45:14.855086 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.855060    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966912"
Sep 07 10:45:14.855283 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.855061    2577 scope.go:117] "RemoveContainer" containerID="fc454af22186bd310df6ec032f8499709033569e522b0f1fbab9cd29b2d82416"
Sep 07 10:45:14.856478 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:14.856389    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7750-139678922966912" event={"ID":"5f5aa6f2-3a55-459c-8a2d-997d874c3ae6","Type":"ContainerStarted","Data":"cb26e53d88e46a1bcb76f37a2e52710a920e181c56f7abc9e67d2b07471807df"}
Sep 07 10:45:15.864447 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:15.864412    2577 generic.go:356] "Generic (PLEG): container finished" podID="5f5aa6f2-3a55-459c-8a2d-997d874c3ae6" containerID="75bb874df66d9482de3237b329d358a7a3db34b168f4aa73d63236fe7b1bb3a5" exitCode=0
Sep 07 10:45:15.864910 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:15.864470    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7750-139678922966912" event={"ID":"5f5aa6f2-3a55-459c-8a2d-997d874c3ae6","Type":"ContainerDied","Data":"75bb874df66d9482de3237b329d358a7a3db34b168f4aa73d63236fe7b1bb3a5"}
Sep 07 10:45:17.010237 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:17.010194    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966912"
Sep 07 10:45:17.033365 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:17.033318    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7750-139678922966912_5f5aa6f2-3a55-459c-8a2d-997d874c3ae6/test-curl-7750-139678922966912/0.log"
Sep 07 10:45:17.059830 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:17.059801    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7750-139678922966912"]
Sep 07 10:45:17.063770 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:17.063744    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7750-139678922966912"]
Sep 07 10:45:17.114795 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:17.114755    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd5w6\" (UniqueName: \"kubernetes.io/projected/5f5aa6f2-3a55-459c-8a2d-997d874c3ae6-kube-api-access-gd5w6\") pod \"5f5aa6f2-3a55-459c-8a2d-997d874c3ae6\" (UID: \"5f5aa6f2-3a55-459c-8a2d-997d874c3ae6\") "
Sep 07 10:45:17.116922 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:17.116891    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f5aa6f2-3a55-459c-8a2d-997d874c3ae6-kube-api-access-gd5w6" (OuterVolumeSpecName: "kube-api-access-gd5w6") pod "5f5aa6f2-3a55-459c-8a2d-997d874c3ae6" (UID: "5f5aa6f2-3a55-459c-8a2d-997d874c3ae6"). InnerVolumeSpecName "kube-api-access-gd5w6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:45:17.215560 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:17.215525    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd5w6\" (UniqueName: \"kubernetes.io/projected/5f5aa6f2-3a55-459c-8a2d-997d874c3ae6-kube-api-access-gd5w6\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:45:17.772108 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:17.772008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:45:17.878484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:17.878453    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb26e53d88e46a1bcb76f37a2e52710a920e181c56f7abc9e67d2b07471807df"
Sep 07 10:45:17.878484 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:17.878478    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7750-139678922966912"
Sep 07 10:45:17.888191 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:45:17.888164    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7750-139678922966912\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="5f5aa6f2-3a55-459c-8a2d-997d874c3ae6" pod="opendatahub/test-curl-7750-139678922966912"
Sep 07 10:45:18.069638 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:18.069555    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:45:18.237251 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:18.237210    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:45:18.785817 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:18.785784    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f5aa6f2-3a55-459c-8a2d-997d874c3ae6" path="/var/lib/kubelet/pods/5f5aa6f2-3a55-459c-8a2d-997d874c3ae6/volumes"
Sep 07 10:45:21.170170 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:21.170103    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:45:21.260300 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:21.260265    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:45:22.300201 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.300115    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"]
Sep 07 10:45:22.301329 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.301149    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f5aa6f2-3a55-459c-8a2d-997d874c3ae6" containerName="test-curl-7750-139678922966912"
Sep 07 10:45:22.301537 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.301521    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f5aa6f2-3a55-459c-8a2d-997d874c3ae6" containerName="test-curl-7750-139678922966912"
Sep 07 10:45:22.302029 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.302014    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f5aa6f2-3a55-459c-8a2d-997d874c3ae6" containerName="test-curl-7750-139678922966912"
Sep 07 10:45:22.308910 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.308071    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:22.311361 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.311334    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2700585f-serving-cert\""
Sep 07 10:45:22.312473 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.312442    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"]
Sep 07 10:45:22.336679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.336643    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:45:22.365091 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.365047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf71c2e0-a21c-464f-b26e-550c37e7eb35-maas-api-tls\") pod \"maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:22.365351 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.365237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x92dk\" (UniqueName: \"kubernetes.io/projected/cf71c2e0-a21c-464f-b26e-550c37e7eb35-kube-api-access-x92dk\") pod \"maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:22.365351 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.365299    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf71c2e0-a21c-464f-b26e-550c37e7eb35-openshift-service-ca\") pod \"maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:22.365467 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.365376    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf71c2e0-a21c-464f-b26e-550c37e7eb35-metrics-certs\") pod \"maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:22.440588 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.440546    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:45:22.466716 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.466673    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x92dk\" (UniqueName: \"kubernetes.io/projected/cf71c2e0-a21c-464f-b26e-550c37e7eb35-kube-api-access-x92dk\") pod \"maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:22.466924 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.466731    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf71c2e0-a21c-464f-b26e-550c37e7eb35-openshift-service-ca\") pod \"maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:22.466924 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.466768    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf71c2e0-a21c-464f-b26e-550c37e7eb35-metrics-certs\") pod \"maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:22.466924 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.466812    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf71c2e0-a21c-464f-b26e-550c37e7eb35-maas-api-tls\") pod \"maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:22.466924 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:45:22.466917    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-serving-cert: secret "maas-api-e2e-mt-2700585f-serving-cert" not found
Sep 07 10:45:22.467183 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:45:22.466987    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf71c2e0-a21c-464f-b26e-550c37e7eb35-maas-api-tls podName:cf71c2e0-a21c-464f-b26e-550c37e7eb35 nodeName:}" failed. No retries permitted until 2026-09-07 10:45:22.966970021 +0000 UTC m=+1526.701144232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cf71c2e0-a21c-464f-b26e-550c37e7eb35-maas-api-tls") pod "maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n" (UID: "cf71c2e0-a21c-464f-b26e-550c37e7eb35") : secret "maas-api-e2e-mt-2700585f-serving-cert" not found
Sep 07 10:45:22.467541 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.467514    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf71c2e0-a21c-464f-b26e-550c37e7eb35-openshift-service-ca\") pod \"maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:22.469588 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.469568    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf71c2e0-a21c-464f-b26e-550c37e7eb35-metrics-certs\") pod \"maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:22.476233 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.476203    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x92dk\" (UniqueName: \"kubernetes.io/projected/cf71c2e0-a21c-464f-b26e-550c37e7eb35-kube-api-access-x92dk\") pod \"maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:22.555682 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.555596    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:45:22.960236 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.960201    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:45:22.971505 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.971467    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf71c2e0-a21c-464f-b26e-550c37e7eb35-maas-api-tls\") pod \"maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:22.975205 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:22.974512    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf71c2e0-a21c-464f-b26e-550c37e7eb35-maas-api-tls\") pod \"maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:23.062257 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:23.062209    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:45:23.222914 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:23.222834    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:23.503173 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:23.503122    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"]
Sep 07 10:45:23.504610 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:45:23.504578    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf71c2e0_a21c_464f_b26e_550c37e7eb35.slice/crio-7bfc2ccbeb1fd6fea4ea0dd5dce12838a9580ce4fd7ed3fe120b1f97b275e424 WatchSource:0}: Error finding container 7bfc2ccbeb1fd6fea4ea0dd5dce12838a9580ce4fd7ed3fe120b1f97b275e424: Status 404 returned error can't find the container with id 7bfc2ccbeb1fd6fea4ea0dd5dce12838a9580ce4fd7ed3fe120b1f97b275e424
Sep 07 10:45:23.916691 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:23.916605    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n" event={"ID":"cf71c2e0-a21c-464f-b26e-550c37e7eb35","Type":"ContainerStarted","Data":"7bfc2ccbeb1fd6fea4ea0dd5dce12838a9580ce4fd7ed3fe120b1f97b275e424"}
Sep 07 10:45:24.926180 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:24.926124    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n" event={"ID":"cf71c2e0-a21c-464f-b26e-550c37e7eb35","Type":"ContainerStarted","Data":"ab1d2768f2b359c77782d9a83ccb26f384cffe27b899f617d818ec943a486601"}
Sep 07 10:45:24.926555 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:24.926269    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:24.945260 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:24.945213    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n" podStartSLOduration=2.433770428 podStartE2EDuration="2.94519749s" totalImagesPullingTime="511.427062ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:45:24.943038136 +0000 UTC m=+1528.677212368" watchObservedRunningTime="2026-09-07 10:45:24.94519749 +0000 UTC m=+1528.679371723"
Sep 07 10:45:30.937443 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:30.937414    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:45:31.145360 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:31.145322    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:45:37.336732 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.336701    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"]
Sep 07 10:45:37.340515 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.340496    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.343325 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.343303    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-659410\"/\"openshift-service-ca.crt\""
Sep 07 10:45:37.344351 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.344329    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-659410\"/\"kube-root-ca.crt\""
Sep 07 10:45:37.344422 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.344381    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-659410\"/\"auth-test-model-659410-kserve-self-signed-certs\""
Sep 07 10:45:37.344463 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.344437    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-659410\"/\"default-dockercfg-8q89w\""
Sep 07 10:45:37.349225 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.349203    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"]
Sep 07 10:45:37.366724 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.366696    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:45:37.407877 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.407845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-home\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.407877 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.407881    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e772c27-7fa8-4cfa-8924-5a61825b5e30-tls-certs\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.408101 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.407904    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-tmp-dir\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.408101 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.408011    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-model-cache\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.408101 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.408063    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-dshm\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.408101 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.408085    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckt4v\" (UniqueName: \"kubernetes.io/projected/4e772c27-7fa8-4cfa-8924-5a61825b5e30-kube-api-access-ckt4v\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.508940 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.508905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-tmp-dir\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.509168 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.508959    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-model-cache\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.509168 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.508987    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-dshm\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.509168 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.509003    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckt4v\" (UniqueName: \"kubernetes.io/projected/4e772c27-7fa8-4cfa-8924-5a61825b5e30-kube-api-access-ckt4v\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.509342 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.509196    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-home\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.509342 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.509233    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e772c27-7fa8-4cfa-8924-5a61825b5e30-tls-certs\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.509464 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.509390    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-tmp-dir\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.509464 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.509439    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-model-cache\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.509590 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.509565    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-home\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.511402 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.511372    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-dshm\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.511721 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.511704    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e772c27-7fa8-4cfa-8924-5a61825b5e30-tls-certs\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.517792 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.517773    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckt4v\" (UniqueName: \"kubernetes.io/projected/4e772c27-7fa8-4cfa-8924-5a61825b5e30-kube-api-access-ckt4v\") pod \"auth-test-model-659410-kserve-7c44b57b7-rnp56\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") " pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.651225 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.651105    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:37.929157 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:37.929097    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"]
Sep 07 10:45:37.934949 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:45:37.934920    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e772c27_7fa8_4cfa_8924_5a61825b5e30.slice/crio-b72414ab010d70c411e47899648f8e7f4939c52326038b2d488ea70ed3da4b36 WatchSource:0}: Error finding container b72414ab010d70c411e47899648f8e7f4939c52326038b2d488ea70ed3da4b36: Status 404 returned error can't find the container with id b72414ab010d70c411e47899648f8e7f4939c52326038b2d488ea70ed3da4b36
Sep 07 10:45:38.018064 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:38.017987    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56" event={"ID":"4e772c27-7fa8-4cfa-8924-5a61825b5e30","Type":"ContainerStarted","Data":"c59de918b5e80885b322dead3f56b527459df01125853b6f2a53acb5b9a5c514"}
Sep 07 10:45:38.018064 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:38.018036    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56" event={"ID":"4e772c27-7fa8-4cfa-8924-5a61825b5e30","Type":"ContainerStarted","Data":"b72414ab010d70c411e47899648f8e7f4939c52326038b2d488ea70ed3da4b36"}
Sep 07 10:45:38.037800 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:38.037744    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56" podStartSLOduration=1.037728425 podStartE2EDuration="1.037728425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:45:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:45:38.034803458 +0000 UTC m=+1541.768977691" watchObservedRunningTime="2026-09-07 10:45:38.037728425 +0000 UTC m=+1541.771902657"
Sep 07 10:45:44.246597 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:44.246554    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:45:46.267272 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:46.267239    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:45:47.651811 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:47.651776    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:47.652240 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:47.651826    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:47.664311 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:47.664287    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:48.098201 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:48.098173    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:45:57.836239 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:45:57.836200    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:46:00.795554 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:00.795513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:46:02.059862 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:02.059825    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:46:02.783192 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:02.783159    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:46:09.859654 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:09.859606    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:46:09.960338 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:09.960300    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:46:17.277405 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:17.277369    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:46:19.332814 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:19.332779    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:46:36.940221 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:36.940182    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:46:42.497944 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:42.497910    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:46:57.339836 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:57.339750    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:46:57.844220 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:57.844155    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"]
Sep 07 10:46:57.844451 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:57.844419    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n" podUID="cf71c2e0-a21c-464f-b26e-550c37e7eb35" containerName="maas-api" containerID="cri-o://ab1d2768f2b359c77782d9a83ccb26f384cffe27b899f617d818ec943a486601" gracePeriod=30
Sep 07 10:46:57.882524 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:57.882486    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:46:57.943285 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:57.943240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:46:58.013506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.013467    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:46:58.099200 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.099091    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:46:58.127931 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.127900    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:46:58.162337 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.162305    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:46:58.252673 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.252636    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf71c2e0-a21c-464f-b26e-550c37e7eb35-openshift-service-ca\") pod \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") "
Sep 07 10:46:58.252862 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.252690    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf71c2e0-a21c-464f-b26e-550c37e7eb35-metrics-certs\") pod \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") "
Sep 07 10:46:58.252862 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.252742    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf71c2e0-a21c-464f-b26e-550c37e7eb35-maas-api-tls\") pod \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") "
Sep 07 10:46:58.252862 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.252764    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x92dk\" (UniqueName: \"kubernetes.io/projected/cf71c2e0-a21c-464f-b26e-550c37e7eb35-kube-api-access-x92dk\") pod \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\" (UID: \"cf71c2e0-a21c-464f-b26e-550c37e7eb35\") "
Sep 07 10:46:58.253030 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.253006    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf71c2e0-a21c-464f-b26e-550c37e7eb35-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cf71c2e0-a21c-464f-b26e-550c37e7eb35" (UID: "cf71c2e0-a21c-464f-b26e-550c37e7eb35"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:46:58.253193 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.253168    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf71c2e0-a21c-464f-b26e-550c37e7eb35-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:46:58.255109 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.255081    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf71c2e0-a21c-464f-b26e-550c37e7eb35-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cf71c2e0-a21c-464f-b26e-550c37e7eb35" (UID: "cf71c2e0-a21c-464f-b26e-550c37e7eb35"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:46:58.255109 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.255100    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf71c2e0-a21c-464f-b26e-550c37e7eb35-kube-api-access-x92dk" (OuterVolumeSpecName: "kube-api-access-x92dk") pod "cf71c2e0-a21c-464f-b26e-550c37e7eb35" (UID: "cf71c2e0-a21c-464f-b26e-550c37e7eb35"). InnerVolumeSpecName "kube-api-access-x92dk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:46:58.255294 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.255228    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf71c2e0-a21c-464f-b26e-550c37e7eb35-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cf71c2e0-a21c-464f-b26e-550c37e7eb35" (UID: "cf71c2e0-a21c-464f-b26e-550c37e7eb35"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:46:58.354624 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.354539    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf71c2e0-a21c-464f-b26e-550c37e7eb35-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:46:58.354624 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.354568    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf71c2e0-a21c-464f-b26e-550c37e7eb35-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:46:58.354624 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.354577    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x92dk\" (UniqueName: \"kubernetes.io/projected/cf71c2e0-a21c-464f-b26e-550c37e7eb35-kube-api-access-x92dk\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:46:58.506481 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.506444    2577 generic.go:356] "Generic (PLEG): container finished" podID="cf71c2e0-a21c-464f-b26e-550c37e7eb35" containerID="ab1d2768f2b359c77782d9a83ccb26f384cffe27b899f617d818ec943a486601" exitCode=0
Sep 07 10:46:58.506655 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.506508    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"
Sep 07 10:46:58.506655 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.506530    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n" event={"ID":"cf71c2e0-a21c-464f-b26e-550c37e7eb35","Type":"ContainerDied","Data":"ab1d2768f2b359c77782d9a83ccb26f384cffe27b899f617d818ec943a486601"}
Sep 07 10:46:58.506655 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.506609    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n" event={"ID":"cf71c2e0-a21c-464f-b26e-550c37e7eb35","Type":"ContainerDied","Data":"7bfc2ccbeb1fd6fea4ea0dd5dce12838a9580ce4fd7ed3fe120b1f97b275e424"}
Sep 07 10:46:58.506655 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.506633    2577 scope.go:117] "RemoveContainer" containerID="ab1d2768f2b359c77782d9a83ccb26f384cffe27b899f617d818ec943a486601"
Sep 07 10:46:58.516555 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.516537    2577 scope.go:117] "RemoveContainer" containerID="ab1d2768f2b359c77782d9a83ccb26f384cffe27b899f617d818ec943a486601"
Sep 07 10:46:58.516857 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:46:58.516839    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab1d2768f2b359c77782d9a83ccb26f384cffe27b899f617d818ec943a486601\": container with ID starting with ab1d2768f2b359c77782d9a83ccb26f384cffe27b899f617d818ec943a486601 not found: ID does not exist" containerID="ab1d2768f2b359c77782d9a83ccb26f384cffe27b899f617d818ec943a486601"
Sep 07 10:46:58.516915 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.516870    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab1d2768f2b359c77782d9a83ccb26f384cffe27b899f617d818ec943a486601"} err="failed to get container status \"ab1d2768f2b359c77782d9a83ccb26f384cffe27b899f617d818ec943a486601\": rpc error: code = NotFound desc = could not find container \"ab1d2768f2b359c77782d9a83ccb26f384cffe27b899f617d818ec943a486601\": container with ID starting with ab1d2768f2b359c77782d9a83ccb26f384cffe27b899f617d818ec943a486601 not found: ID does not exist"
Sep 07 10:46:58.530098 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.530072    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"]
Sep 07 10:46:58.535487 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.535463    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2700585f-cfd84c57f-fdn8n"]
Sep 07 10:46:58.785181 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:46:58.785121    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf71c2e0-a21c-464f-b26e-550c37e7eb35" path="/var/lib/kubelet/pods/cf71c2e0-a21c-464f-b26e-550c37e7eb35/volumes"
Sep 07 10:47:02.146636 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:02.146597    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:07.160677 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:07.160639    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:16.942593 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:16.942550    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:47:21.400225 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:21.400184    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:33.529665 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:33.529630    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:47:43.690064 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:43.690016    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:44.496437 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:44.496396    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:48.831450 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:48.831418    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:47:50.092083 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:50.092044    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:50.158503 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:50.158468    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:50.377982 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:50.377894    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:51.439105 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:51.439066    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7753-c44b08"]
Sep 07 10:47:51.439702 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:51.439681    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf71c2e0-a21c-464f-b26e-550c37e7eb35" containerName="maas-api"
Sep 07 10:47:51.439787 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:51.439705    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf71c2e0-a21c-464f-b26e-550c37e7eb35" containerName="maas-api"
Sep 07 10:47:51.439840 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:51.439807    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf71c2e0-a21c-464f-b26e-550c37e7eb35" containerName="maas-api"
Sep 07 10:47:51.443345 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:51.443324    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7753-c44b08"
Sep 07 10:47:51.446227 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:51.446195    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-xwqv8\""
Sep 07 10:47:51.448509 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:51.448485    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7753-c44b08"]
Sep 07 10:47:51.546650 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:51.546611    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxnr7\" (UniqueName: \"kubernetes.io/projected/6224f89c-73e6-4081-9428-227056b48d37-kube-api-access-fxnr7\") pod \"mt-curl-7753-c44b08\" (UID: \"6224f89c-73e6-4081-9428-227056b48d37\") " pod="opendatahub/mt-curl-7753-c44b08"
Sep 07 10:47:51.647967 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:51.647925    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxnr7\" (UniqueName: \"kubernetes.io/projected/6224f89c-73e6-4081-9428-227056b48d37-kube-api-access-fxnr7\") pod \"mt-curl-7753-c44b08\" (UID: \"6224f89c-73e6-4081-9428-227056b48d37\") " pod="opendatahub/mt-curl-7753-c44b08"
Sep 07 10:47:51.656308 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:51.656275    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxnr7\" (UniqueName: \"kubernetes.io/projected/6224f89c-73e6-4081-9428-227056b48d37-kube-api-access-fxnr7\") pod \"mt-curl-7753-c44b08\" (UID: \"6224f89c-73e6-4081-9428-227056b48d37\") " pod="opendatahub/mt-curl-7753-c44b08"
Sep 07 10:47:51.753747 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:51.753711    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7753-c44b08"
Sep 07 10:47:51.892625 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:51.892573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:52.027519 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:52.027429    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:52.047000 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:52.046968    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7753-c44b08"]
Sep 07 10:47:52.048230 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:47:52.048203    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6224f89c_73e6_4081_9428_227056b48d37.slice/crio-2c8ae4d153a7416383d53e0322e512100b2e992228f2c4cb58927e9f8f102ba1 WatchSource:0}: Error finding container 2c8ae4d153a7416383d53e0322e512100b2e992228f2c4cb58927e9f8f102ba1: Status 404 returned error can't find the container with id 2c8ae4d153a7416383d53e0322e512100b2e992228f2c4cb58927e9f8f102ba1
Sep 07 10:47:52.049645 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:52.049628    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 10:47:52.119763 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:52.119728    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:52.777535 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:52.777502    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:52.849572 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:52.849525    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7753-c44b08" event={"ID":"6224f89c-73e6-4081-9428-227056b48d37","Type":"ContainerStarted","Data":"46d0b363dfcd172a3a94c0482286e3e1bbff80b3d60387c360cb529543c64395"}
Sep 07 10:47:52.849746 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:52.849576    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7753-c44b08" event={"ID":"6224f89c-73e6-4081-9428-227056b48d37","Type":"ContainerStarted","Data":"2c8ae4d153a7416383d53e0322e512100b2e992228f2c4cb58927e9f8f102ba1"}
Sep 07 10:47:52.871529 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:52.871496    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:52.872518 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:52.872471    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/mt-curl-7753-c44b08" podStartSLOduration=1.184508411 podStartE2EDuration="1.872455272s" totalImagesPullingTime="687.946861ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:47:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:47:52.869315339 +0000 UTC m=+1676.603489572" watchObservedRunningTime="2026-09-07 10:47:52.872455272 +0000 UTC m=+1676.606629594"
Sep 07 10:47:52.965458 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:52.965422    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:52.965994 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:52.965971    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7753-c44b08_6224f89c-73e6-4081-9428-227056b48d37/mt-curl-7753-c44b08/0.log"
Sep 07 10:47:53.857179 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:53.857121    2577 generic.go:356] "Generic (PLEG): container finished" podID="6224f89c-73e6-4081-9428-227056b48d37" containerID="46d0b363dfcd172a3a94c0482286e3e1bbff80b3d60387c360cb529543c64395" exitCode=0
Sep 07 10:47:53.857179 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:53.857172    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7753-c44b08" event={"ID":"6224f89c-73e6-4081-9428-227056b48d37","Type":"ContainerDied","Data":"46d0b363dfcd172a3a94c0482286e3e1bbff80b3d60387c360cb529543c64395"}
Sep 07 10:47:54.498824 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:54.498791    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:54.992548 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:54.992526    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7753-c44b08"
Sep 07 10:47:55.030071 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:55.030038    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7753-c44b08"]
Sep 07 10:47:55.034498 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:55.034470    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7753-c44b08"]
Sep 07 10:47:55.076049 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:55.076013    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxnr7\" (UniqueName: \"kubernetes.io/projected/6224f89c-73e6-4081-9428-227056b48d37-kube-api-access-fxnr7\") pod \"6224f89c-73e6-4081-9428-227056b48d37\" (UID: \"6224f89c-73e6-4081-9428-227056b48d37\") "
Sep 07 10:47:55.078361 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:55.078333    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6224f89c-73e6-4081-9428-227056b48d37-kube-api-access-fxnr7" (OuterVolumeSpecName: "kube-api-access-fxnr7") pod "6224f89c-73e6-4081-9428-227056b48d37" (UID: "6224f89c-73e6-4081-9428-227056b48d37"). InnerVolumeSpecName "kube-api-access-fxnr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:47:55.177426 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:55.177343    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxnr7\" (UniqueName: \"kubernetes.io/projected/6224f89c-73e6-4081-9428-227056b48d37-kube-api-access-fxnr7\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:47:55.870692 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:55.870658    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7753-c44b08"
Sep 07 10:47:55.870692 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:55.870693    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c8ae4d153a7416383d53e0322e512100b2e992228f2c4cb58927e9f8f102ba1"
Sep 07 10:47:55.880723 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:47:55.880693    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7753-c44b08\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="6224f89c-73e6-4081-9428-227056b48d37" pod="opendatahub/mt-curl-7753-c44b08"
Sep 07 10:47:56.503905 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:56.503862    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:47:56.784055 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:47:56.783956    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7753-c44b08\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="6224f89c-73e6-4081-9428-227056b48d37" pod="opendatahub/mt-curl-7753-c44b08"
Sep 07 10:47:56.786640 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:56.786610    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6224f89c-73e6-4081-9428-227056b48d37" path="/var/lib/kubelet/pods/6224f89c-73e6-4081-9428-227056b48d37/volumes"
Sep 07 10:47:57.525506 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:57.525475    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"]
Sep 07 10:47:57.544003 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:47:57.543940    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:00.249676 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:00.249640    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:00.373646 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:00.373609    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:01.037613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:01.037573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:06.868569 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:06.868526    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:06.950274 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:06.950243    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:07.046426 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:07.046393    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:07.145658 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:07.145560    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:07.437977 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:07.437889    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:48:07.850880 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:07.850845    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:12.546483 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.546414    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56" podUID="4e772c27-7fa8-4cfa-8924-5a61825b5e30" containerName="main" containerID="cri-o://c59de918b5e80885b322dead3f56b527459df01125853b6f2a53acb5b9a5c514" gracePeriod=45
Sep 07 10:48:12.814513 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.814488    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:48:12.950580 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.950544    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckt4v\" (UniqueName: \"kubernetes.io/projected/4e772c27-7fa8-4cfa-8924-5a61825b5e30-kube-api-access-ckt4v\") pod \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") "
Sep 07 10:48:12.950778 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.950594    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-tmp-dir\") pod \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") "
Sep 07 10:48:12.950778 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.950650    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e772c27-7fa8-4cfa-8924-5a61825b5e30-tls-certs\") pod \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") "
Sep 07 10:48:12.950778 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.950686    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-dshm\") pod \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") "
Sep 07 10:48:12.950778 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.950711    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-model-cache\") pod \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") "
Sep 07 10:48:12.950987 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.950805    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-home\") pod \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\" (UID: \"4e772c27-7fa8-4cfa-8924-5a61825b5e30\") "
Sep 07 10:48:12.951038 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.950957    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e772c27-7fa8-4cfa-8924-5a61825b5e30" (UID: "4e772c27-7fa8-4cfa-8924-5a61825b5e30"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:48:12.951104 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.951031    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e772c27-7fa8-4cfa-8924-5a61825b5e30" (UID: "4e772c27-7fa8-4cfa-8924-5a61825b5e30"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:48:12.951184 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.951121    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-home" (OuterVolumeSpecName: "home") pod "4e772c27-7fa8-4cfa-8924-5a61825b5e30" (UID: "4e772c27-7fa8-4cfa-8924-5a61825b5e30"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:48:12.951184 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.951172    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-model-cache\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:48:12.951268 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.951210    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-tmp-dir\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:48:12.953086 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.953054    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e772c27-7fa8-4cfa-8924-5a61825b5e30-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e772c27-7fa8-4cfa-8924-5a61825b5e30" (UID: "4e772c27-7fa8-4cfa-8924-5a61825b5e30"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:48:12.953495 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.953474    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-dshm" (OuterVolumeSpecName: "dshm") pod "4e772c27-7fa8-4cfa-8924-5a61825b5e30" (UID: "4e772c27-7fa8-4cfa-8924-5a61825b5e30"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:48:12.953495 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.953486    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e772c27-7fa8-4cfa-8924-5a61825b5e30-kube-api-access-ckt4v" (OuterVolumeSpecName: "kube-api-access-ckt4v") pod "4e772c27-7fa8-4cfa-8924-5a61825b5e30" (UID: "4e772c27-7fa8-4cfa-8924-5a61825b5e30"). InnerVolumeSpecName "kube-api-access-ckt4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:48:12.975814 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.975775    2577 generic.go:356] "Generic (PLEG): container finished" podID="4e772c27-7fa8-4cfa-8924-5a61825b5e30" containerID="c59de918b5e80885b322dead3f56b527459df01125853b6f2a53acb5b9a5c514" exitCode=0
Sep 07 10:48:12.975995 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.975833    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"
Sep 07 10:48:12.975995 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.975861    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56" event={"ID":"4e772c27-7fa8-4cfa-8924-5a61825b5e30","Type":"ContainerDied","Data":"c59de918b5e80885b322dead3f56b527459df01125853b6f2a53acb5b9a5c514"}
Sep 07 10:48:12.975995 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.975911    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56" event={"ID":"4e772c27-7fa8-4cfa-8924-5a61825b5e30","Type":"ContainerDied","Data":"b72414ab010d70c411e47899648f8e7f4939c52326038b2d488ea70ed3da4b36"}
Sep 07 10:48:12.975995 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.975930    2577 scope.go:117] "RemoveContainer" containerID="c59de918b5e80885b322dead3f56b527459df01125853b6f2a53acb5b9a5c514"
Sep 07 10:48:12.986353 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.986327    2577 scope.go:117] "RemoveContainer" containerID="c59de918b5e80885b322dead3f56b527459df01125853b6f2a53acb5b9a5c514"
Sep 07 10:48:12.986716 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:48:12.986679    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c59de918b5e80885b322dead3f56b527459df01125853b6f2a53acb5b9a5c514\": container with ID starting with c59de918b5e80885b322dead3f56b527459df01125853b6f2a53acb5b9a5c514 not found: ID does not exist" containerID="c59de918b5e80885b322dead3f56b527459df01125853b6f2a53acb5b9a5c514"
Sep 07 10:48:12.986831 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:12.986719    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c59de918b5e80885b322dead3f56b527459df01125853b6f2a53acb5b9a5c514"} err="failed to get container status \"c59de918b5e80885b322dead3f56b527459df01125853b6f2a53acb5b9a5c514\": rpc error: code = NotFound desc = could not find container \"c59de918b5e80885b322dead3f56b527459df01125853b6f2a53acb5b9a5c514\": container with ID starting with c59de918b5e80885b322dead3f56b527459df01125853b6f2a53acb5b9a5c514 not found: ID does not exist"
Sep 07 10:48:13.001104 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:13.001063    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"]
Sep 07 10:48:13.007860 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:13.007837    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-659410/auth-test-model-659410-kserve-7c44b57b7-rnp56"]
Sep 07 10:48:13.052702 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:13.052621    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-home\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:48:13.052702 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:13.052648    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckt4v\" (UniqueName: \"kubernetes.io/projected/4e772c27-7fa8-4cfa-8924-5a61825b5e30-kube-api-access-ckt4v\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:48:13.052702 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:13.052660    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e772c27-7fa8-4cfa-8924-5a61825b5e30-tls-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:48:13.052702 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:13.052669    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e772c27-7fa8-4cfa-8924-5a61825b5e30-dshm\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:48:14.785679 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:14.785646    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e772c27-7fa8-4cfa-8924-5a61825b5e30" path="/var/lib/kubelet/pods/4e772c27-7fa8-4cfa-8924-5a61825b5e30/volumes"
Sep 07 10:48:17.699907 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:17.699818    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:18.004473 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:18.004362    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:18.331443 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:18.331355    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:48:25.544718 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:25.544677    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:29.438273 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:29.438236    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:48:38.951198 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:38.951159    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:41.701892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:41.701848    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:44.799892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:44.799854    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:44.863285 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:44.863255    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:45.135211 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:45.135107    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:45.438801 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:45.438714    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:45.805661 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:45.805628    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:48.733901 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:48.733870    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:48.863552 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:48.863519    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:49.873172 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:49.873124    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"]
Sep 07 10:48:49.873742 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:49.873721    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e772c27-7fa8-4cfa-8924-5a61825b5e30" containerName="main"
Sep 07 10:48:49.873833 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:49.873744    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e772c27-7fa8-4cfa-8924-5a61825b5e30" containerName="main"
Sep 07 10:48:49.873833 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:49.873785    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6224f89c-73e6-4081-9428-227056b48d37" containerName="mt-curl-7753-c44b08"
Sep 07 10:48:49.873833 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:49.873794    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6224f89c-73e6-4081-9428-227056b48d37" containerName="mt-curl-7753-c44b08"
Sep 07 10:48:49.873991 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:49.873902    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e772c27-7fa8-4cfa-8924-5a61825b5e30" containerName="main"
Sep 07 10:48:49.873991 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:49.873916    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6224f89c-73e6-4081-9428-227056b48d37" containerName="mt-curl-7753-c44b08"
Sep 07 10:48:49.877521 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:49.877500    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:49.880230 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:49.880211    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-d4e2c0-serving-cert\""
Sep 07 10:48:49.888238 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:49.888213    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"]
Sep 07 10:48:49.913897 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:49.913866    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:49.987508 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:49.987473    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz46r\" (UniqueName: \"kubernetes.io/projected/9e696a48-a475-4b1e-a602-6251ac9304d9-kube-api-access-wz46r\") pod \"maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:49.987660 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:49.987532    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e696a48-a475-4b1e-a602-6251ac9304d9-maas-api-tls\") pod \"maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:49.987660 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:49.987576    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e696a48-a475-4b1e-a602-6251ac9304d9-openshift-service-ca\") pod \"maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:49.987660 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:49.987592    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e696a48-a475-4b1e-a602-6251ac9304d9-metrics-certs\") pod \"maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:50.013711 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:50.013676    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:50.088616 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:50.088579    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz46r\" (UniqueName: \"kubernetes.io/projected/9e696a48-a475-4b1e-a602-6251ac9304d9-kube-api-access-wz46r\") pod \"maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:50.088796 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:50.088651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e696a48-a475-4b1e-a602-6251ac9304d9-maas-api-tls\") pod \"maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:50.088854 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:50.088808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e696a48-a475-4b1e-a602-6251ac9304d9-openshift-service-ca\") pod \"maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:50.088854 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:50.088843    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e696a48-a475-4b1e-a602-6251ac9304d9-metrics-certs\") pod \"maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:50.089526 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:50.089504    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e696a48-a475-4b1e-a602-6251ac9304d9-openshift-service-ca\") pod \"maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:50.091269 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:50.091250    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e696a48-a475-4b1e-a602-6251ac9304d9-metrics-certs\") pod \"maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:50.091348 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:50.091308    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e696a48-a475-4b1e-a602-6251ac9304d9-maas-api-tls\") pod \"maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:50.097466 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:50.097443    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz46r\" (UniqueName: \"kubernetes.io/projected/9e696a48-a475-4b1e-a602-6251ac9304d9-kube-api-access-wz46r\") pod \"maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:50.135777 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:50.135682    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:50.191579 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:50.191544    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:50.427800 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:50.427708    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:50.467681 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:50.467651    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"]
Sep 07 10:48:50.469681 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:48:50.469651    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e696a48_a475_4b1e_a602_6251ac9304d9.slice/crio-07a9d120f313afa1d923ad9d2ab8261c488c7e2880fb8f215f518ee3273aa115 WatchSource:0}: Error finding container 07a9d120f313afa1d923ad9d2ab8261c488c7e2880fb8f215f518ee3273aa115: Status 404 returned error can't find the container with id 07a9d120f313afa1d923ad9d2ab8261c488c7e2880fb8f215f518ee3273aa115
Sep 07 10:48:50.522796 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:50.522753    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:48:51.211425 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:51.211385    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4" event={"ID":"9e696a48-a475-4b1e-a602-6251ac9304d9","Type":"ContainerStarted","Data":"57059056c4829226af6e026346bc7df97d1533dbca0a76da9a95473a4a293029"}
Sep 07 10:48:51.211823 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:51.211454    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4" event={"ID":"9e696a48-a475-4b1e-a602-6251ac9304d9","Type":"ContainerStarted","Data":"07a9d120f313afa1d923ad9d2ab8261c488c7e2880fb8f215f518ee3273aa115"}
Sep 07 10:48:51.211823 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:51.211483    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:51.232190 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:51.232084    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4" podStartSLOduration=1.734775924 podStartE2EDuration="2.23206875s" totalImagesPullingTime="497.292826ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:48:51.229070344 +0000 UTC m=+1734.963244623" watchObservedRunningTime="2026-09-07 10:48:51.23206875 +0000 UTC m=+1734.966243000"
Sep 07 10:48:57.223017 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:57.222990    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:48:59.537414 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:48:59.537367    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:49:10.706304 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:10.706268    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:11.393149 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:11.393096    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:13.633748 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:13.633703    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:14.086571 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:14.086537    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:14.388267 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:14.388125    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:14.546644 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:14.546602    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:17.950207 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:17.950164    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:18.040704 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:18.040668    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:19.070350 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.070320    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"]
Sep 07 10:49:19.074980 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.074951    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:19.077778 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.077744    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-4f3dee-serving-cert\""
Sep 07 10:49:19.086545 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.086518    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"]
Sep 07 10:49:19.104735 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.104702    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:19.156892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.156852    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13df89c-3c04-4007-a80d-f66afa0e6360-maas-api-tls\") pod \"maas-api-e2e-api-b-4f3dee-6fc479447-nq77t\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:19.156892 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.156888    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13df89c-3c04-4007-a80d-f66afa0e6360-openshift-service-ca\") pod \"maas-api-e2e-api-b-4f3dee-6fc479447-nq77t\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:19.157107 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.156983    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgmtx\" (UniqueName: \"kubernetes.io/projected/f13df89c-3c04-4007-a80d-f66afa0e6360-kube-api-access-tgmtx\") pod \"maas-api-e2e-api-b-4f3dee-6fc479447-nq77t\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:19.157107 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.157034    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13df89c-3c04-4007-a80d-f66afa0e6360-metrics-certs\") pod \"maas-api-e2e-api-b-4f3dee-6fc479447-nq77t\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:19.206567 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.206532    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:19.257792 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.257763    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13df89c-3c04-4007-a80d-f66afa0e6360-openshift-service-ca\") pod \"maas-api-e2e-api-b-4f3dee-6fc479447-nq77t\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:19.257949 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.257822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgmtx\" (UniqueName: \"kubernetes.io/projected/f13df89c-3c04-4007-a80d-f66afa0e6360-kube-api-access-tgmtx\") pod \"maas-api-e2e-api-b-4f3dee-6fc479447-nq77t\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:19.257949 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.257845    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13df89c-3c04-4007-a80d-f66afa0e6360-metrics-certs\") pod \"maas-api-e2e-api-b-4f3dee-6fc479447-nq77t\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:19.257949 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.257896    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13df89c-3c04-4007-a80d-f66afa0e6360-maas-api-tls\") pod \"maas-api-e2e-api-b-4f3dee-6fc479447-nq77t\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:19.258476 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.258452    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13df89c-3c04-4007-a80d-f66afa0e6360-openshift-service-ca\") pod \"maas-api-e2e-api-b-4f3dee-6fc479447-nq77t\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:19.260531 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.260509    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13df89c-3c04-4007-a80d-f66afa0e6360-maas-api-tls\") pod \"maas-api-e2e-api-b-4f3dee-6fc479447-nq77t\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:19.260621 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.260574    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13df89c-3c04-4007-a80d-f66afa0e6360-metrics-certs\") pod \"maas-api-e2e-api-b-4f3dee-6fc479447-nq77t\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:19.265877 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.265859    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgmtx\" (UniqueName: \"kubernetes.io/projected/f13df89c-3c04-4007-a80d-f66afa0e6360-kube-api-access-tgmtx\") pod \"maas-api-e2e-api-b-4f3dee-6fc479447-nq77t\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:19.307275 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.307235    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:19.335344 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.335264    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:49:19.388649 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.388619    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:19.648659 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.648581    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:19.678278 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.675624    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"]
Sep 07 10:49:19.737274 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:19.737240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:20.396444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:20.396356    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t" event={"ID":"f13df89c-3c04-4007-a80d-f66afa0e6360","Type":"ContainerStarted","Data":"d622b29470cf49dfc753a140335311ba4a136416d94502961817f3011344fbde"}
Sep 07 10:49:20.396444 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:20.396406    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t" event={"ID":"f13df89c-3c04-4007-a80d-f66afa0e6360","Type":"ContainerStarted","Data":"ffa0af37685041138e664900a56d7706aa5d25f2a2e36e86f97b7d046043e904"}
Sep 07 10:49:20.396846 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:20.396454    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:20.415252 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:20.415199    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t" podStartSLOduration=1.057086313 podStartE2EDuration="1.4151837s" totalImagesPullingTime="358.097387ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:49:20.414189756 +0000 UTC m=+1764.148363980" watchObservedRunningTime="2026-09-07 10:49:20.4151837 +0000 UTC m=+1764.149357941"
Sep 07 10:49:22.573672 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:22.573633    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7753-bbca40"]
Sep 07 10:49:22.578302 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:22.578265    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7753-bbca40"
Sep 07 10:49:22.580934 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:22.580905    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-xwqv8\""
Sep 07 10:49:22.583936 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:22.583916    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7753-bbca40"]
Sep 07 10:49:22.693841 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:22.693804    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkdqs\" (UniqueName: \"kubernetes.io/projected/61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07-kube-api-access-vkdqs\") pod \"mt-curl-7753-bbca40\" (UID: \"61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07\") " pod="opendatahub/mt-curl-7753-bbca40"
Sep 07 10:49:22.795187 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:22.795151    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkdqs\" (UniqueName: \"kubernetes.io/projected/61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07-kube-api-access-vkdqs\") pod \"mt-curl-7753-bbca40\" (UID: \"61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07\") " pod="opendatahub/mt-curl-7753-bbca40"
Sep 07 10:49:22.805453 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:22.805423    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkdqs\" (UniqueName: \"kubernetes.io/projected/61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07-kube-api-access-vkdqs\") pod \"mt-curl-7753-bbca40\" (UID: \"61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07\") " pod="opendatahub/mt-curl-7753-bbca40"
Sep 07 10:49:22.893527 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:22.893444    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7753-bbca40"
Sep 07 10:49:23.166680 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:23.166648    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7753-bbca40"]
Sep 07 10:49:23.168087 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:49:23.168061    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61efdbc3_17c8_4b9c_8cf4_1c5b224a9d07.slice/crio-f87d5d43101df29e99a05c2de84574059af0fc6a7f704db8b33392aad5f2bfaf WatchSource:0}: Error finding container f87d5d43101df29e99a05c2de84574059af0fc6a7f704db8b33392aad5f2bfaf: Status 404 returned error can't find the container with id f87d5d43101df29e99a05c2de84574059af0fc6a7f704db8b33392aad5f2bfaf
Sep 07 10:49:23.415469 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:23.415384    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7753-bbca40" event={"ID":"61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07","Type":"ContainerStarted","Data":"f87d5d43101df29e99a05c2de84574059af0fc6a7f704db8b33392aad5f2bfaf"}
Sep 07 10:49:24.423960 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:24.423925    2577 generic.go:356] "Generic (PLEG): container finished" podID="61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07" containerID="0c7737c7725cecbe57ce2e4f1a4cfffe5bb704816be3c3f4a708e60c5976f5b4" exitCode=0
Sep 07 10:49:24.424356 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:24.424013    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7753-bbca40" event={"ID":"61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07","Type":"ContainerDied","Data":"0c7737c7725cecbe57ce2e4f1a4cfffe5bb704816be3c3f4a708e60c5976f5b4"}
Sep 07 10:49:25.573473 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.573449    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7753-bbca40"
Sep 07 10:49:25.592938 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.592912    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7753-bbca40_61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07/mt-curl-7753-bbca40/0.log"
Sep 07 10:49:25.615184 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.615158    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7753-bbca40"]
Sep 07 10:49:25.617576 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.617558    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7753-bbca40"]
Sep 07 10:49:25.622600 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.622583    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkdqs\" (UniqueName: \"kubernetes.io/projected/61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07-kube-api-access-vkdqs\") pod \"61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07\" (UID: \"61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07\") "
Sep 07 10:49:25.624782 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.624758    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07-kube-api-access-vkdqs" (OuterVolumeSpecName: "kube-api-access-vkdqs") pod "61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07" (UID: "61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07"). InnerVolumeSpecName "kube-api-access-vkdqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:49:25.723848 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.723820    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkdqs\" (UniqueName: \"kubernetes.io/projected/61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07-kube-api-access-vkdqs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:49:25.739199 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.739172    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7753-bdf54d"]
Sep 07 10:49:25.739612 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.739598    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07" containerName="mt-curl-7753-bbca40"
Sep 07 10:49:25.739668 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.739614    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07" containerName="mt-curl-7753-bbca40"
Sep 07 10:49:25.739706 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.739687    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07" containerName="mt-curl-7753-bbca40"
Sep 07 10:49:25.742687 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.742671    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7753-bdf54d"
Sep 07 10:49:25.756162 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.751675    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7753-bdf54d"]
Sep 07 10:49:25.825375 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.825327    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jvgm\" (UniqueName: \"kubernetes.io/projected/6765b414-05f4-410c-8054-6ac24eb2cde0-kube-api-access-2jvgm\") pod \"mt-curl-7753-bdf54d\" (UID: \"6765b414-05f4-410c-8054-6ac24eb2cde0\") " pod="opendatahub/mt-curl-7753-bdf54d"
Sep 07 10:49:25.926645 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.926608    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jvgm\" (UniqueName: \"kubernetes.io/projected/6765b414-05f4-410c-8054-6ac24eb2cde0-kube-api-access-2jvgm\") pod \"mt-curl-7753-bdf54d\" (UID: \"6765b414-05f4-410c-8054-6ac24eb2cde0\") " pod="opendatahub/mt-curl-7753-bdf54d"
Sep 07 10:49:25.935107 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:25.935074    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jvgm\" (UniqueName: \"kubernetes.io/projected/6765b414-05f4-410c-8054-6ac24eb2cde0-kube-api-access-2jvgm\") pod \"mt-curl-7753-bdf54d\" (UID: \"6765b414-05f4-410c-8054-6ac24eb2cde0\") " pod="opendatahub/mt-curl-7753-bdf54d"
Sep 07 10:49:26.059972 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:26.059868    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7753-bdf54d"
Sep 07 10:49:26.336074 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:26.336044    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7753-bdf54d"]
Sep 07 10:49:26.336792 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:49:26.336769    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6765b414_05f4_410c_8054_6ac24eb2cde0.slice/crio-70eab715baa414d855d4bf41ec4c1a28e3fbe4c0d0dbb4ddcd3ebff0c4ba33b3 WatchSource:0}: Error finding container 70eab715baa414d855d4bf41ec4c1a28e3fbe4c0d0dbb4ddcd3ebff0c4ba33b3: Status 404 returned error can't find the container with id 70eab715baa414d855d4bf41ec4c1a28e3fbe4c0d0dbb4ddcd3ebff0c4ba33b3
Sep 07 10:49:26.408180 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:26.408152    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:49:26.438914 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:26.438881    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f87d5d43101df29e99a05c2de84574059af0fc6a7f704db8b33392aad5f2bfaf"
Sep 07 10:49:26.439248 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:26.438908    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7753-bbca40"
Sep 07 10:49:26.440510 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:26.440482    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7753-bdf54d" event={"ID":"6765b414-05f4-410c-8054-6ac24eb2cde0","Type":"ContainerStarted","Data":"70eab715baa414d855d4bf41ec4c1a28e3fbe4c0d0dbb4ddcd3ebff0c4ba33b3"}
Sep 07 10:49:26.453942 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:49:26.453909    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7753-bbca40\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07" pod="opendatahub/mt-curl-7753-bbca40"
Sep 07 10:49:26.783747 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:49:26.783714    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7753-bbca40\" is forbidden: User \"system:node:ip-10-0-128-166.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-166.ec2.internal' and this object" podUID="61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07" pod="opendatahub/mt-curl-7753-bbca40"
Sep 07 10:49:26.785793 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:26.785768    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07" path="/var/lib/kubelet/pods/61efdbc3-17c8-4b9c-8cf4-1c5b224a9d07/volumes"
Sep 07 10:49:27.448609 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:27.448577    2577 generic.go:356] "Generic (PLEG): container finished" podID="6765b414-05f4-410c-8054-6ac24eb2cde0" containerID="0274e8f5790ff6e500069a6e45c18b6bb6d3266e32b1542a20db26cb21e83ae3" exitCode=0
Sep 07 10:49:27.448856 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:27.448659    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7753-bdf54d" event={"ID":"6765b414-05f4-410c-8054-6ac24eb2cde0","Type":"ContainerDied","Data":"0274e8f5790ff6e500069a6e45c18b6bb6d3266e32b1542a20db26cb21e83ae3"}
Sep 07 10:49:28.603852 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:28.603826    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7753-bdf54d"
Sep 07 10:49:28.625180 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:28.625153    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7753-bdf54d_6765b414-05f4-410c-8054-6ac24eb2cde0/mt-curl-7753-bdf54d/0.log"
Sep 07 10:49:28.648422 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:28.648393    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7753-bdf54d"]
Sep 07 10:49:28.651293 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:28.651269    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jvgm\" (UniqueName: \"kubernetes.io/projected/6765b414-05f4-410c-8054-6ac24eb2cde0-kube-api-access-2jvgm\") pod \"6765b414-05f4-410c-8054-6ac24eb2cde0\" (UID: \"6765b414-05f4-410c-8054-6ac24eb2cde0\") "
Sep 07 10:49:28.652716 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:28.652696    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7753-bdf54d"]
Sep 07 10:49:28.653467 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:28.653436    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6765b414-05f4-410c-8054-6ac24eb2cde0-kube-api-access-2jvgm" (OuterVolumeSpecName: "kube-api-access-2jvgm") pod "6765b414-05f4-410c-8054-6ac24eb2cde0" (UID: "6765b414-05f4-410c-8054-6ac24eb2cde0"). InnerVolumeSpecName "kube-api-access-2jvgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:49:28.752196 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:28.752087    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jvgm\" (UniqueName: \"kubernetes.io/projected/6765b414-05f4-410c-8054-6ac24eb2cde0-kube-api-access-2jvgm\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:49:28.785829 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:28.785794    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6765b414-05f4-410c-8054-6ac24eb2cde0" path="/var/lib/kubelet/pods/6765b414-05f4-410c-8054-6ac24eb2cde0/volumes"
Sep 07 10:49:29.468685 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:29.468652    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7753-bdf54d"
Sep 07 10:49:29.468898 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:29.468687    2577 scope.go:117] "RemoveContainer" containerID="0274e8f5790ff6e500069a6e45c18b6bb6d3266e32b1542a20db26cb21e83ae3"
Sep 07 10:49:30.745570 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:30.745537    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:32.603328 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:32.603292    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:32.962658 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:32.962623    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:37.234787 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:37.234751    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:49:37.560386 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:37.560303    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:40.129241 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:40.129208    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:43.326586 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:43.326552    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:47.228414 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:47.228375    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:52.196289 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:52.196248    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:49:53.230656 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:53.230621    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:49:56.897659 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:56.897618    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:49:56.898956 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:49:56.898934    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:50:00.982962 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:00.982919    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:02.641695 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:02.641651    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"]
Sep 07 10:50:02.642206 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:02.641897    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4" podUID="9e696a48-a475-4b1e-a602-6251ac9304d9" containerName="maas-api" containerID="cri-o://57059056c4829226af6e026346bc7df97d1533dbca0a76da9a95473a4a293029" gracePeriod=30
Sep 07 10:50:02.669735 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:02.669704    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:02.771936 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:02.771901    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:02.871115 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:02.871074    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:02.889410 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:02.889383    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:50:02.971941 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:02.971814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:02.972779 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:02.972415    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e696a48-a475-4b1e-a602-6251ac9304d9-maas-api-tls\") pod \"9e696a48-a475-4b1e-a602-6251ac9304d9\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") "
Sep 07 10:50:02.972779 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:02.972455    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e696a48-a475-4b1e-a602-6251ac9304d9-openshift-service-ca\") pod \"9e696a48-a475-4b1e-a602-6251ac9304d9\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") "
Sep 07 10:50:02.972779 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:02.972491    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wz46r\" (UniqueName: \"kubernetes.io/projected/9e696a48-a475-4b1e-a602-6251ac9304d9-kube-api-access-wz46r\") pod \"9e696a48-a475-4b1e-a602-6251ac9304d9\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") "
Sep 07 10:50:02.972779 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:02.972569    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e696a48-a475-4b1e-a602-6251ac9304d9-metrics-certs\") pod \"9e696a48-a475-4b1e-a602-6251ac9304d9\" (UID: \"9e696a48-a475-4b1e-a602-6251ac9304d9\") "
Sep 07 10:50:02.973179 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:02.973050    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e696a48-a475-4b1e-a602-6251ac9304d9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9e696a48-a475-4b1e-a602-6251ac9304d9" (UID: "9e696a48-a475-4b1e-a602-6251ac9304d9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:50:02.975216 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:02.975187    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e696a48-a475-4b1e-a602-6251ac9304d9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9e696a48-a475-4b1e-a602-6251ac9304d9" (UID: "9e696a48-a475-4b1e-a602-6251ac9304d9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:50:02.975468 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:02.975443    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e696a48-a475-4b1e-a602-6251ac9304d9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9e696a48-a475-4b1e-a602-6251ac9304d9" (UID: "9e696a48-a475-4b1e-a602-6251ac9304d9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:50:02.975601 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:02.975528    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e696a48-a475-4b1e-a602-6251ac9304d9-kube-api-access-wz46r" (OuterVolumeSpecName: "kube-api-access-wz46r") pod "9e696a48-a475-4b1e-a602-6251ac9304d9" (UID: "9e696a48-a475-4b1e-a602-6251ac9304d9"). InnerVolumeSpecName "kube-api-access-wz46r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:50:03.075070 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:03.074893    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e696a48-a475-4b1e-a602-6251ac9304d9-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:50:03.075070 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:03.074941    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e696a48-a475-4b1e-a602-6251ac9304d9-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:50:03.075070 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:03.074956    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e696a48-a475-4b1e-a602-6251ac9304d9-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:50:03.075070 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:03.074970    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wz46r\" (UniqueName: \"kubernetes.io/projected/9e696a48-a475-4b1e-a602-6251ac9304d9-kube-api-access-wz46r\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:50:03.291095 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:03.290994    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:03.695613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:03.695535    2577 generic.go:356] "Generic (PLEG): container finished" podID="9e696a48-a475-4b1e-a602-6251ac9304d9" containerID="57059056c4829226af6e026346bc7df97d1533dbca0a76da9a95473a4a293029" exitCode=0
Sep 07 10:50:03.695613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:03.695593    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"
Sep 07 10:50:03.695613 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:03.695590    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4" event={"ID":"9e696a48-a475-4b1e-a602-6251ac9304d9","Type":"ContainerDied","Data":"57059056c4829226af6e026346bc7df97d1533dbca0a76da9a95473a4a293029"}
Sep 07 10:50:03.696149 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:03.695669    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4" event={"ID":"9e696a48-a475-4b1e-a602-6251ac9304d9","Type":"ContainerDied","Data":"07a9d120f313afa1d923ad9d2ab8261c488c7e2880fb8f215f518ee3273aa115"}
Sep 07 10:50:03.696149 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:03.695690    2577 scope.go:117] "RemoveContainer" containerID="57059056c4829226af6e026346bc7df97d1533dbca0a76da9a95473a4a293029"
Sep 07 10:50:03.705938 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:03.705921    2577 scope.go:117] "RemoveContainer" containerID="57059056c4829226af6e026346bc7df97d1533dbca0a76da9a95473a4a293029"
Sep 07 10:50:03.706229 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:50:03.706199    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57059056c4829226af6e026346bc7df97d1533dbca0a76da9a95473a4a293029\": container with ID starting with 57059056c4829226af6e026346bc7df97d1533dbca0a76da9a95473a4a293029 not found: ID does not exist" containerID="57059056c4829226af6e026346bc7df97d1533dbca0a76da9a95473a4a293029"
Sep 07 10:50:03.706301 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:03.706238    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57059056c4829226af6e026346bc7df97d1533dbca0a76da9a95473a4a293029"} err="failed to get container status \"57059056c4829226af6e026346bc7df97d1533dbca0a76da9a95473a4a293029\": rpc error: code = NotFound desc = could not find container \"57059056c4829226af6e026346bc7df97d1533dbca0a76da9a95473a4a293029\": container with ID starting with 57059056c4829226af6e026346bc7df97d1533dbca0a76da9a95473a4a293029 not found: ID does not exist"
Sep 07 10:50:03.719139 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:03.719098    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"]
Sep 07 10:50:03.721584 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:03.721563    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d4e2c0-7454cd5695-q6ph4"]
Sep 07 10:50:04.785929 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:04.785892    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e696a48-a475-4b1e-a602-6251ac9304d9" path="/var/lib/kubelet/pods/9e696a48-a475-4b1e-a602-6251ac9304d9/volumes"
Sep 07 10:50:07.177836 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:07.177798    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:07.530270 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:07.530236    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:50:09.071846 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:09.071809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:09.172205 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:09.172167    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:09.320481 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:09.320450    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:09.896549 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:09.896497    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:16.752526 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:16.752484    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:23.035151 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:23.035087    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:50:25.194381 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.186919    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"]
Sep 07 10:50:25.194381 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.187313    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t" podUID="f13df89c-3c04-4007-a80d-f66afa0e6360" containerName="maas-api" containerID="cri-o://d622b29470cf49dfc753a140335311ba4a136416d94502961817f3011344fbde" gracePeriod=30
Sep 07 10:50:25.212181 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.212086    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:25.313068 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.313034    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:25.412656 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.412626    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:25.450088 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.450067    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:50:25.483535 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.483482    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:25.592028 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.591981    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13df89c-3c04-4007-a80d-f66afa0e6360-maas-api-tls\") pod \"f13df89c-3c04-4007-a80d-f66afa0e6360\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") "
Sep 07 10:50:25.592261 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.592070    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13df89c-3c04-4007-a80d-f66afa0e6360-openshift-service-ca\") pod \"f13df89c-3c04-4007-a80d-f66afa0e6360\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") "
Sep 07 10:50:25.592261 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.592102    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13df89c-3c04-4007-a80d-f66afa0e6360-metrics-certs\") pod \"f13df89c-3c04-4007-a80d-f66afa0e6360\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") "
Sep 07 10:50:25.592261 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.592186    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgmtx\" (UniqueName: \"kubernetes.io/projected/f13df89c-3c04-4007-a80d-f66afa0e6360-kube-api-access-tgmtx\") pod \"f13df89c-3c04-4007-a80d-f66afa0e6360\" (UID: \"f13df89c-3c04-4007-a80d-f66afa0e6360\") "
Sep 07 10:50:25.592505 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.592468    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f13df89c-3c04-4007-a80d-f66afa0e6360-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f13df89c-3c04-4007-a80d-f66afa0e6360" (UID: "f13df89c-3c04-4007-a80d-f66afa0e6360"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:50:25.594412 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.594387    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13df89c-3c04-4007-a80d-f66afa0e6360-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f13df89c-3c04-4007-a80d-f66afa0e6360" (UID: "f13df89c-3c04-4007-a80d-f66afa0e6360"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:50:25.594520 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.594435    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f13df89c-3c04-4007-a80d-f66afa0e6360-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f13df89c-3c04-4007-a80d-f66afa0e6360" (UID: "f13df89c-3c04-4007-a80d-f66afa0e6360"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:50:25.594520 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.594437    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f13df89c-3c04-4007-a80d-f66afa0e6360-kube-api-access-tgmtx" (OuterVolumeSpecName: "kube-api-access-tgmtx") pod "f13df89c-3c04-4007-a80d-f66afa0e6360" (UID: "f13df89c-3c04-4007-a80d-f66afa0e6360"). InnerVolumeSpecName "kube-api-access-tgmtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:50:25.693845 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.693810    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f13df89c-3c04-4007-a80d-f66afa0e6360-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:50:25.693845 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.693839    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f13df89c-3c04-4007-a80d-f66afa0e6360-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:50:25.693845 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.693851    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f13df89c-3c04-4007-a80d-f66afa0e6360-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:50:25.694078 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.693859    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgmtx\" (UniqueName: \"kubernetes.io/projected/f13df89c-3c04-4007-a80d-f66afa0e6360-kube-api-access-tgmtx\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:50:25.779065 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.778977    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:25.831723 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.831688    2577 generic.go:356] "Generic (PLEG): container finished" podID="f13df89c-3c04-4007-a80d-f66afa0e6360" containerID="d622b29470cf49dfc753a140335311ba4a136416d94502961817f3011344fbde" exitCode=0
Sep 07 10:50:25.831898 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.831771    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"
Sep 07 10:50:25.831898 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.831814    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t" event={"ID":"f13df89c-3c04-4007-a80d-f66afa0e6360","Type":"ContainerDied","Data":"d622b29470cf49dfc753a140335311ba4a136416d94502961817f3011344fbde"}
Sep 07 10:50:25.831898 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.831867    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t" event={"ID":"f13df89c-3c04-4007-a80d-f66afa0e6360","Type":"ContainerDied","Data":"ffa0af37685041138e664900a56d7706aa5d25f2a2e36e86f97b7d046043e904"}
Sep 07 10:50:25.831898 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.831889    2577 scope.go:117] "RemoveContainer" containerID="d622b29470cf49dfc753a140335311ba4a136416d94502961817f3011344fbde"
Sep 07 10:50:25.845069 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.843942    2577 scope.go:117] "RemoveContainer" containerID="d622b29470cf49dfc753a140335311ba4a136416d94502961817f3011344fbde"
Sep 07 10:50:25.845225 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:50:25.845159    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d622b29470cf49dfc753a140335311ba4a136416d94502961817f3011344fbde\": container with ID starting with d622b29470cf49dfc753a140335311ba4a136416d94502961817f3011344fbde not found: ID does not exist" containerID="d622b29470cf49dfc753a140335311ba4a136416d94502961817f3011344fbde"
Sep 07 10:50:25.845225 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.845191    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d622b29470cf49dfc753a140335311ba4a136416d94502961817f3011344fbde"} err="failed to get container status \"d622b29470cf49dfc753a140335311ba4a136416d94502961817f3011344fbde\": rpc error: code = NotFound desc = could not find container \"d622b29470cf49dfc753a140335311ba4a136416d94502961817f3011344fbde\": container with ID starting with d622b29470cf49dfc753a140335311ba4a136416d94502961817f3011344fbde not found: ID does not exist"
Sep 07 10:50:25.854653 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.854630    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"]
Sep 07 10:50:25.858246 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:25.858223    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4f3dee-6fc479447-nq77t"]
Sep 07 10:50:26.785158 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:26.785112    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f13df89c-3c04-4007-a80d-f66afa0e6360" path="/var/lib/kubelet/pods/f13df89c-3c04-4007-a80d-f66afa0e6360/volumes"
Sep 07 10:50:31.639394 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:31.639359    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:31.709411 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:31.709379    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:31.910106 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:31.910012    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:32.400029 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:32.399993    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:39.532079 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:39.532033    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:50:42.703236 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:42.703202    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"]
Sep 07 10:50:42.703725 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:42.703579    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv" podUID="28dfc33b-4247-463b-88c6-76dfb11a3925" containerName="maas-api" containerID="cri-o://469a95822f3e0c0a1c2ae306a27dce3424c5006cbab74856a1582b41a7147b67" gracePeriod=30
Sep 07 10:50:42.729778 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:42.729729    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:42.816389 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:42.816355    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:42.914425 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:42.914391    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:42.941067 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:42.941041    2577 generic.go:356] "Generic (PLEG): container finished" podID="28dfc33b-4247-463b-88c6-76dfb11a3925" containerID="469a95822f3e0c0a1c2ae306a27dce3424c5006cbab74856a1582b41a7147b67" exitCode=0
Sep 07 10:50:42.941238 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:42.941125    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv" event={"ID":"28dfc33b-4247-463b-88c6-76dfb11a3925","Type":"ContainerDied","Data":"469a95822f3e0c0a1c2ae306a27dce3424c5006cbab74856a1582b41a7147b67"}
Sep 07 10:50:42.960125 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:42.960075    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:50:42.974452 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:42.974424    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:43.064638 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.064606    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28dfc33b-4247-463b-88c6-76dfb11a3925-maas-api-tls\") pod \"28dfc33b-4247-463b-88c6-76dfb11a3925\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") "
Sep 07 10:50:43.064837 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.064649    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28dfc33b-4247-463b-88c6-76dfb11a3925-openshift-service-ca\") pod \"28dfc33b-4247-463b-88c6-76dfb11a3925\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") "
Sep 07 10:50:43.064837 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.064678    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28dfc33b-4247-463b-88c6-76dfb11a3925-metrics-certs\") pod \"28dfc33b-4247-463b-88c6-76dfb11a3925\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") "
Sep 07 10:50:43.064837 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.064701    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dt6wt\" (UniqueName: \"kubernetes.io/projected/28dfc33b-4247-463b-88c6-76dfb11a3925-kube-api-access-dt6wt\") pod \"28dfc33b-4247-463b-88c6-76dfb11a3925\" (UID: \"28dfc33b-4247-463b-88c6-76dfb11a3925\") "
Sep 07 10:50:43.065059 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.065033    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28dfc33b-4247-463b-88c6-76dfb11a3925-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28dfc33b-4247-463b-88c6-76dfb11a3925" (UID: "28dfc33b-4247-463b-88c6-76dfb11a3925"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:50:43.067189 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.067153    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28dfc33b-4247-463b-88c6-76dfb11a3925-kube-api-access-dt6wt" (OuterVolumeSpecName: "kube-api-access-dt6wt") pod "28dfc33b-4247-463b-88c6-76dfb11a3925" (UID: "28dfc33b-4247-463b-88c6-76dfb11a3925"). InnerVolumeSpecName "kube-api-access-dt6wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:50:43.067455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.067427    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28dfc33b-4247-463b-88c6-76dfb11a3925-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "28dfc33b-4247-463b-88c6-76dfb11a3925" (UID: "28dfc33b-4247-463b-88c6-76dfb11a3925"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:50:43.067547 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.067450    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28dfc33b-4247-463b-88c6-76dfb11a3925-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28dfc33b-4247-463b-88c6-76dfb11a3925" (UID: "28dfc33b-4247-463b-88c6-76dfb11a3925"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:50:43.166057 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.166022    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28dfc33b-4247-463b-88c6-76dfb11a3925-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:50:43.166057 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.166051    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28dfc33b-4247-463b-88c6-76dfb11a3925-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:50:43.166057 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.166062    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28dfc33b-4247-463b-88c6-76dfb11a3925-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:50:43.166315 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.166070    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt6wt\" (UniqueName: \"kubernetes.io/projected/28dfc33b-4247-463b-88c6-76dfb11a3925-kube-api-access-dt6wt\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:50:43.324497 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.324401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:43.949980 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.949954    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"
Sep 07 10:50:43.950411 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.949984    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv" event={"ID":"28dfc33b-4247-463b-88c6-76dfb11a3925","Type":"ContainerDied","Data":"a9282e8b6839745591011c9762e7bbadb6036f535c738c84f875502f8dc6c221"}
Sep 07 10:50:43.950411 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.950030    2577 scope.go:117] "RemoveContainer" containerID="469a95822f3e0c0a1c2ae306a27dce3424c5006cbab74856a1582b41a7147b67"
Sep 07 10:50:43.973120 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.973083    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"]
Sep 07 10:50:43.978223 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:43.978197    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a158df-c497fc98f-2skdv"]
Sep 07 10:50:44.785952 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:44.785912    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28dfc33b-4247-463b-88c6-76dfb11a3925" path="/var/lib/kubelet/pods/28dfc33b-4247-463b-88c6-76dfb11a3925/volumes"
Sep 07 10:50:49.096756 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:49.096715    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:49.197198 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:49.197165    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:49.351943 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:49.351864    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:50:49.776313 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:49.776268    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:50:49.978983 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:50:49.978948    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:00.318768 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.318725    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"]
Sep 07 10:51:00.319248 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.318988    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn" podUID="1aebccc8-116d-474f-a504-6e8d4906635c" containerName="maas-api" containerID="cri-o://cbadc4b3911c191f8f545ad689b00629304452777ad1d8bb2b40eab3ff4dbb51" gracePeriod=30
Sep 07 10:51:00.350371 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.350339    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:00.452951 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.452908    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:00.545561 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.545526    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:00.573511 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.573442    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:51:00.604252 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.604217    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:00.725830 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.725796    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aebccc8-116d-474f-a504-6e8d4906635c-openshift-service-ca\") pod \"1aebccc8-116d-474f-a504-6e8d4906635c\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") "
Sep 07 10:51:00.726014 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.725862    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-maas-api-tls\") pod \"1aebccc8-116d-474f-a504-6e8d4906635c\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") "
Sep 07 10:51:00.726014 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.725886    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgv5r\" (UniqueName: \"kubernetes.io/projected/1aebccc8-116d-474f-a504-6e8d4906635c-kube-api-access-hgv5r\") pod \"1aebccc8-116d-474f-a504-6e8d4906635c\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") "
Sep 07 10:51:00.726014 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.725948    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-metrics-certs\") pod \"1aebccc8-116d-474f-a504-6e8d4906635c\" (UID: \"1aebccc8-116d-474f-a504-6e8d4906635c\") "
Sep 07 10:51:00.726265 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.726237    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aebccc8-116d-474f-a504-6e8d4906635c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1aebccc8-116d-474f-a504-6e8d4906635c" (UID: "1aebccc8-116d-474f-a504-6e8d4906635c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:51:00.728244 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.728213    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1aebccc8-116d-474f-a504-6e8d4906635c" (UID: "1aebccc8-116d-474f-a504-6e8d4906635c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:51:00.728374 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.728274    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1aebccc8-116d-474f-a504-6e8d4906635c" (UID: "1aebccc8-116d-474f-a504-6e8d4906635c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:51:00.728374 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.728290    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aebccc8-116d-474f-a504-6e8d4906635c-kube-api-access-hgv5r" (OuterVolumeSpecName: "kube-api-access-hgv5r") pod "1aebccc8-116d-474f-a504-6e8d4906635c" (UID: "1aebccc8-116d-474f-a504-6e8d4906635c"). InnerVolumeSpecName "kube-api-access-hgv5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:51:00.827413 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.827341    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:51:00.827413 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.827367    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1aebccc8-116d-474f-a504-6e8d4906635c-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:51:00.827413 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.827378    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1aebccc8-116d-474f-a504-6e8d4906635c-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:51:00.827413 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.827386    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgv5r\" (UniqueName: \"kubernetes.io/projected/1aebccc8-116d-474f-a504-6e8d4906635c-kube-api-access-hgv5r\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:51:00.887985 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:00.887934    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:01.066075 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:01.066040    2577 generic.go:356] "Generic (PLEG): container finished" podID="1aebccc8-116d-474f-a504-6e8d4906635c" containerID="cbadc4b3911c191f8f545ad689b00629304452777ad1d8bb2b40eab3ff4dbb51" exitCode=0
Sep 07 10:51:01.066284 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:01.066085    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"
Sep 07 10:51:01.066284 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:01.066124    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn" event={"ID":"1aebccc8-116d-474f-a504-6e8d4906635c","Type":"ContainerDied","Data":"cbadc4b3911c191f8f545ad689b00629304452777ad1d8bb2b40eab3ff4dbb51"}
Sep 07 10:51:01.066284 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:01.066207    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn" event={"ID":"1aebccc8-116d-474f-a504-6e8d4906635c","Type":"ContainerDied","Data":"58fc0c903845094bae5c5d9f07c21118609380cd9939b8221e1376cf2ad6cf38"}
Sep 07 10:51:01.066284 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:01.066229    2577 scope.go:117] "RemoveContainer" containerID="cbadc4b3911c191f8f545ad689b00629304452777ad1d8bb2b40eab3ff4dbb51"
Sep 07 10:51:01.076467 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:01.076437    2577 scope.go:117] "RemoveContainer" containerID="cbadc4b3911c191f8f545ad689b00629304452777ad1d8bb2b40eab3ff4dbb51"
Sep 07 10:51:01.076843 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:51:01.076816    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbadc4b3911c191f8f545ad689b00629304452777ad1d8bb2b40eab3ff4dbb51\": container with ID starting with cbadc4b3911c191f8f545ad689b00629304452777ad1d8bb2b40eab3ff4dbb51 not found: ID does not exist" containerID="cbadc4b3911c191f8f545ad689b00629304452777ad1d8bb2b40eab3ff4dbb51"
Sep 07 10:51:01.076950 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:01.076851    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbadc4b3911c191f8f545ad689b00629304452777ad1d8bb2b40eab3ff4dbb51"} err="failed to get container status \"cbadc4b3911c191f8f545ad689b00629304452777ad1d8bb2b40eab3ff4dbb51\": rpc error: code = NotFound desc = could not find container \"cbadc4b3911c191f8f545ad689b00629304452777ad1d8bb2b40eab3ff4dbb51\": container with ID starting with cbadc4b3911c191f8f545ad689b00629304452777ad1d8bb2b40eab3ff4dbb51 not found: ID does not exist"
Sep 07 10:51:01.085406 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:01.085344    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"]
Sep 07 10:51:01.089537 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:01.089512    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-5e934f-65b98d65d5-4bwtn"]
Sep 07 10:51:02.784924 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:02.784894    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1aebccc8-116d-474f-a504-6e8d4906635c" path="/var/lib/kubelet/pods/1aebccc8-116d-474f-a504-6e8d4906635c/volumes"
Sep 07 10:51:03.141410 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:03.141322    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:04.353616 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:04.353575    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:06.430979 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:06.430944    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:51:06.848069 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:06.848031    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:06.917339 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:06.917301    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:07.117782 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:07.117705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:18.080109 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:18.080002    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:18.739385 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:18.739348    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:51:24.369332 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:24.369302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:25.327983 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:25.327942    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:25.747502 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:25.747462    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:26.041993 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:26.041914    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:26.181864 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:26.181830    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:28.728353 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:28.728321    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:28.918305 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:28.918265    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:29.738551 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.738514    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:51:29.848478 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.848444    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"]
Sep 07 10:51:29.848854 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.848842    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aebccc8-116d-474f-a504-6e8d4906635c" containerName="maas-api"
Sep 07 10:51:29.848906 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.848855    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aebccc8-116d-474f-a504-6e8d4906635c" containerName="maas-api"
Sep 07 10:51:29.848906 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.848873    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f13df89c-3c04-4007-a80d-f66afa0e6360" containerName="maas-api"
Sep 07 10:51:29.848906 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.848878    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f13df89c-3c04-4007-a80d-f66afa0e6360" containerName="maas-api"
Sep 07 10:51:29.848906 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.848885    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28dfc33b-4247-463b-88c6-76dfb11a3925" containerName="maas-api"
Sep 07 10:51:29.848906 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.848891    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="28dfc33b-4247-463b-88c6-76dfb11a3925" containerName="maas-api"
Sep 07 10:51:29.848906 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.848897    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6765b414-05f4-410c-8054-6ac24eb2cde0" containerName="mt-curl-7753-bdf54d"
Sep 07 10:51:29.848906 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.848903    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6765b414-05f4-410c-8054-6ac24eb2cde0" containerName="mt-curl-7753-bdf54d"
Sep 07 10:51:29.849182 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.848920    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e696a48-a475-4b1e-a602-6251ac9304d9" containerName="maas-api"
Sep 07 10:51:29.849182 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.848925    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e696a48-a475-4b1e-a602-6251ac9304d9" containerName="maas-api"
Sep 07 10:51:29.849182 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.848985    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e696a48-a475-4b1e-a602-6251ac9304d9" containerName="maas-api"
Sep 07 10:51:29.849182 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.848993    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="28dfc33b-4247-463b-88c6-76dfb11a3925" containerName="maas-api"
Sep 07 10:51:29.849182 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.848999    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aebccc8-116d-474f-a504-6e8d4906635c" containerName="maas-api"
Sep 07 10:51:29.849182 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.849005    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f13df89c-3c04-4007-a80d-f66afa0e6360" containerName="maas-api"
Sep 07 10:51:29.849182 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.849010    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6765b414-05f4-410c-8054-6ac24eb2cde0" containerName="mt-curl-7753-bdf54d"
Sep 07 10:51:29.852247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.852212    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:29.855577 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.855551    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-bef02f-serving-cert\""
Sep 07 10:51:29.870604 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.870569    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"]
Sep 07 10:51:29.880188 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.880122    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f631bdfa-6ee7-442a-b736-40ae95ce0605-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:29.880462 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.880439    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f631bdfa-6ee7-442a-b736-40ae95ce0605-metrics-certs\") pod \"maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:29.880656 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.880637    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49dbl\" (UniqueName: \"kubernetes.io/projected/f631bdfa-6ee7-442a-b736-40ae95ce0605-kube-api-access-49dbl\") pod \"maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:29.880914 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.880894    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f631bdfa-6ee7-442a-b736-40ae95ce0605-maas-api-tls\") pod \"maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:29.886612 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.886585    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:29.959347 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.959306    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:29.981782 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.981750    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f631bdfa-6ee7-442a-b736-40ae95ce0605-maas-api-tls\") pod \"maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:29.981941 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.981809    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f631bdfa-6ee7-442a-b736-40ae95ce0605-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:29.981941 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.981852    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f631bdfa-6ee7-442a-b736-40ae95ce0605-metrics-certs\") pod \"maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:29.981941 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.981869    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49dbl\" (UniqueName: \"kubernetes.io/projected/f631bdfa-6ee7-442a-b736-40ae95ce0605-kube-api-access-49dbl\") pod \"maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:29.981941 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:51:29.981903    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-serving-cert: secret "maas-api-e2e-ipp-a-bef02f-serving-cert" not found
Sep 07 10:51:29.982115 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:51:29.981987    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f631bdfa-6ee7-442a-b736-40ae95ce0605-maas-api-tls podName:f631bdfa-6ee7-442a-b736-40ae95ce0605 nodeName:}" failed. No retries permitted until 2026-09-07 10:51:30.481966107 +0000 UTC m=+1894.216140341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f631bdfa-6ee7-442a-b736-40ae95ce0605-maas-api-tls") pod "maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv" (UID: "f631bdfa-6ee7-442a-b736-40ae95ce0605") : secret "maas-api-e2e-ipp-a-bef02f-serving-cert" not found
Sep 07 10:51:29.982716 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.982647    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f631bdfa-6ee7-442a-b736-40ae95ce0605-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:29.984525 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.984504    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f631bdfa-6ee7-442a-b736-40ae95ce0605-metrics-certs\") pod \"maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:29.995646 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:29.995586    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49dbl\" (UniqueName: \"kubernetes.io/projected/f631bdfa-6ee7-442a-b736-40ae95ce0605-kube-api-access-49dbl\") pod \"maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:30.284612 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:30.284517    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:30.458988 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:30.458955    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:30.486365 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:30.486333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f631bdfa-6ee7-442a-b736-40ae95ce0605-maas-api-tls\") pod \"maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:30.489481 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:30.489448    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f631bdfa-6ee7-442a-b736-40ae95ce0605-maas-api-tls\") pod \"maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:30.550095 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:30.550014    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:30.764345 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:30.764312    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:31.042839 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:31.042810    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"]
Sep 07 10:51:31.044582 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:51:31.044553    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf631bdfa_6ee7_442a_b736_40ae95ce0605.slice/crio-ee5ccbee1cef1ae7ccafea807dd889c2df695f6a8dafaaff9abdd25b1395d7ea WatchSource:0}: Error finding container ee5ccbee1cef1ae7ccafea807dd889c2df695f6a8dafaaff9abdd25b1395d7ea: Status 404 returned error can't find the container with id ee5ccbee1cef1ae7ccafea807dd889c2df695f6a8dafaaff9abdd25b1395d7ea
Sep 07 10:51:31.256367 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:31.256324    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv" event={"ID":"f631bdfa-6ee7-442a-b736-40ae95ce0605","Type":"ContainerStarted","Data":"ee5ccbee1cef1ae7ccafea807dd889c2df695f6a8dafaaff9abdd25b1395d7ea"}
Sep 07 10:51:32.263978 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:32.263941    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv" event={"ID":"f631bdfa-6ee7-442a-b736-40ae95ce0605","Type":"ContainerStarted","Data":"77f9ffb7ec79d234bde270c453cf4eee43afe0268c800773e0be5673c09aa0ee"}
Sep 07 10:51:32.264479 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:32.264009    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:32.282733 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:32.282687    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv" podStartSLOduration=2.9498665920000002 podStartE2EDuration="3.282670149s" totalImagesPullingTime="332.803557ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:51:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:51:32.281935385 +0000 UTC m=+1896.016109613" watchObservedRunningTime="2026-09-07 10:51:32.282670149 +0000 UTC m=+1896.016844426"
Sep 07 10:51:38.275920 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:38.275886    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:51:40.530700 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:40.530656    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:51:53.705993 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:53.705954    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:54.356082 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:54.356040    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:54.660183 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:54.660080    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:54.944932 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:54.944844    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:57.066731 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:57.066689    2577 scope.go:117] "RemoveContainer" containerID="75bb874df66d9482de3237b329d358a7a3db34b168f4aa73d63236fe7b1bb3a5"
Sep 07 10:51:57.077606 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:57.077587    2577 scope.go:117] "RemoveContainer" containerID="45da76da28aec36b48bb5cee548e359d30e1133736f50d36a568d83246c1a2aa"
Sep 07 10:51:58.082303 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:58.082251    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:58.178556 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:58.178523    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:59.197287 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.197252    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"]
Sep 07 10:51:59.200983 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.200960    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:51:59.203377 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.203354    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-738923-serving-cert\""
Sep 07 10:51:59.211457 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.211433    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"]
Sep 07 10:51:59.268900 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.268869    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:59.353711 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.353680    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a539b55e-2b82-488f-8123-956b2b85e91a-metrics-certs\") pod \"maas-api-e2e-ipp-b-738923-797b85b747-zplbf\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:51:59.353888 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.353722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a539b55e-2b82-488f-8123-956b2b85e91a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-738923-797b85b747-zplbf\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:51:59.353888 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.353866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a539b55e-2b82-488f-8123-956b2b85e91a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-738923-797b85b747-zplbf\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:51:59.353972 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.353912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48rd5\" (UniqueName: \"kubernetes.io/projected/a539b55e-2b82-488f-8123-956b2b85e91a-kube-api-access-48rd5\") pod \"maas-api-e2e-ipp-b-738923-797b85b747-zplbf\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:51:59.368998 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.368944    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:59.455025 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.454938    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48rd5\" (UniqueName: \"kubernetes.io/projected/a539b55e-2b82-488f-8123-956b2b85e91a-kube-api-access-48rd5\") pod \"maas-api-e2e-ipp-b-738923-797b85b747-zplbf\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:51:59.455025 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.454981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a539b55e-2b82-488f-8123-956b2b85e91a-metrics-certs\") pod \"maas-api-e2e-ipp-b-738923-797b85b747-zplbf\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:51:59.455025 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.455003    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a539b55e-2b82-488f-8123-956b2b85e91a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-738923-797b85b747-zplbf\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:51:59.455499 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.455074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a539b55e-2b82-488f-8123-956b2b85e91a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-738923-797b85b747-zplbf\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:51:59.455837 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.455812    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a539b55e-2b82-488f-8123-956b2b85e91a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-738923-797b85b747-zplbf\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:51:59.457564 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.457533    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a539b55e-2b82-488f-8123-956b2b85e91a-metrics-certs\") pod \"maas-api-e2e-ipp-b-738923-797b85b747-zplbf\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:51:59.457744 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.457725    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a539b55e-2b82-488f-8123-956b2b85e91a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-738923-797b85b747-zplbf\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:51:59.462702 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.462676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48rd5\" (UniqueName: \"kubernetes.io/projected/a539b55e-2b82-488f-8123-956b2b85e91a-kube-api-access-48rd5\") pod \"maas-api-e2e-ipp-b-738923-797b85b747-zplbf\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:51:59.469546 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.469504    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:59.512069 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.512035    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:51:59.792167 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.792111    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"]
Sep 07 10:51:59.794416 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:51:59.794389    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda539b55e_2b82_488f_8123_956b2b85e91a.slice/crio-b8d471ad29da42b57e218e6d289d807b05d190ca5a6979c970b01f626d1f2097 WatchSource:0}: Error finding container b8d471ad29da42b57e218e6d289d807b05d190ca5a6979c970b01f626d1f2097: Status 404 returned error can't find the container with id b8d471ad29da42b57e218e6d289d807b05d190ca5a6979c970b01f626d1f2097
Sep 07 10:51:59.810663 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.810632    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:51:59.897178 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:51:59.897116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:52:00.437765 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:52:00.437684    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf" event={"ID":"a539b55e-2b82-488f-8123-956b2b85e91a","Type":"ContainerStarted","Data":"46f2f613ece19b236512f773a8833f7d60bac36ecf960e328f2bde55875c04a4"}
Sep 07 10:52:00.437765 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:52:00.437740    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf" event={"ID":"a539b55e-2b82-488f-8123-956b2b85e91a","Type":"ContainerStarted","Data":"b8d471ad29da42b57e218e6d289d807b05d190ca5a6979c970b01f626d1f2097"}
Sep 07 10:52:00.438158 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:52:00.437774    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:52:00.455084 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:52:00.455037    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf" podStartSLOduration=1.104279383 podStartE2EDuration="1.455020835s" totalImagesPullingTime="350.741452ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:51:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:52:00.453787466 +0000 UTC m=+1924.187961699" watchObservedRunningTime="2026-09-07 10:52:00.455020835 +0000 UTC m=+1924.189195068"
Sep 07 10:52:03.137164 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:52:03.137114    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:52:06.449408 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:52:06.449379    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:52:23.056680 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:52:23.056644    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:52:37.156532 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:52:37.156492    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:52:41.039709 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:52:41.039677    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:52:59.846784 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:52:59.846706    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:53:06.417324 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:06.417277    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:06.719496 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:06.719464    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:06.809723 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:06.809684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:09.382811 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:09.382756    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:09.477915 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:09.477877    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:10.499369 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.499338    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"]
Sep 07 10:53:10.503827 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.503804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:10.506391 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.506368    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-39c953-serving-cert\""
Sep 07 10:53:10.514448 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.514423    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"]
Sep 07 10:53:10.532498 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.532471    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:10.575540 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.575511    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:10.575690 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.575546    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:10.575690 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.575580    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bp96\" (UniqueName: \"kubernetes.io/projected/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-kube-api-access-2bp96\") pod \"maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:10.575690 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.575684    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:10.631962 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.631913    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:10.677045 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.677009    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:10.677235 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.677073    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:10.677235 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.677092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:10.677235 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.677126    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bp96\" (UniqueName: \"kubernetes.io/projected/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-kube-api-access-2bp96\") pod \"maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:10.677900 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.677825    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:10.679701 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.679678    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:10.679835 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.679812    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:10.685488 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.685467    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bp96\" (UniqueName: \"kubernetes.io/projected/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-kube-api-access-2bp96\") pod \"maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:10.758688 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.758599    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:10.815903 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:10.815866    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:11.030052 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:11.029946    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:11.096019 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:11.095989    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"]
Sep 07 10:53:11.097664 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:53:11.097624    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod786d1b56_b737_47e6_a3f3_4e00fe5c8f08.slice/crio-a9553d2fd5ec2c9ffa5d4ea0a038c4589cdebb44a70eedd004fe5f49f0964497 WatchSource:0}: Error finding container a9553d2fd5ec2c9ffa5d4ea0a038c4589cdebb44a70eedd004fe5f49f0964497: Status 404 returned error can't find the container with id a9553d2fd5ec2c9ffa5d4ea0a038c4589cdebb44a70eedd004fe5f49f0964497
Sep 07 10:53:11.099078 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:11.099060    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 10:53:11.101500 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:11.101472    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:11.877706 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:11.877611    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t" event={"ID":"786d1b56-b737-47e6-a3f3-4e00fe5c8f08","Type":"ContainerStarted","Data":"36aafe3e657cef42c8f899bfa32f743351ccccbc99688dac2d69ca4fe4d98ad8"}
Sep 07 10:53:11.878189 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:11.877707    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t" event={"ID":"786d1b56-b737-47e6-a3f3-4e00fe5c8f08","Type":"ContainerStarted","Data":"a9553d2fd5ec2c9ffa5d4ea0a038c4589cdebb44a70eedd004fe5f49f0964497"}
Sep 07 10:53:11.878189 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:11.877735    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:11.896252 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:11.896206    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t" podStartSLOduration=1.404139705 podStartE2EDuration="1.89619266s" totalImagesPullingTime="492.052955ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:53:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:53:11.8948904 +0000 UTC m=+1995.629064634" watchObservedRunningTime="2026-09-07 10:53:11.89619266 +0000 UTC m=+1995.630366894"
Sep 07 10:53:17.889109 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:17.889078    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:19.735700 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:19.735659    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:53:27.846867 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:27.846822    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:34.399785 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:34.399749    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:35.550727 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:35.550692    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:41.024122 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.024086    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"]
Sep 07 10:53:41.024602 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.024306    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t" podUID="786d1b56-b737-47e6-a3f3-4e00fe5c8f08" containerName="maas-api" containerID="cri-o://36aafe3e657cef42c8f899bfa32f743351ccccbc99688dac2d69ca4fe4d98ad8" gracePeriod=30
Sep 07 10:53:41.041321 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.041291    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:41.115461 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.115430    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:41.179061 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.179028    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:41.250197 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.250159    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:41.276525 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.276466    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:41.310595 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.310561    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:41.463462 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.463433    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-openshift-service-ca\") pod \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") "
Sep 07 10:53:41.463644 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.463490    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bp96\" (UniqueName: \"kubernetes.io/projected/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-kube-api-access-2bp96\") pod \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") "
Sep 07 10:53:41.463644 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.463613    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-metrics-certs\") pod \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") "
Sep 07 10:53:41.463644 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.463638    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-maas-api-tls\") pod \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\" (UID: \"786d1b56-b737-47e6-a3f3-4e00fe5c8f08\") "
Sep 07 10:53:41.463813 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.463789    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "786d1b56-b737-47e6-a3f3-4e00fe5c8f08" (UID: "786d1b56-b737-47e6-a3f3-4e00fe5c8f08"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:53:41.463960 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.463939    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:53:41.465760 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.465729    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-kube-api-access-2bp96" (OuterVolumeSpecName: "kube-api-access-2bp96") pod "786d1b56-b737-47e6-a3f3-4e00fe5c8f08" (UID: "786d1b56-b737-47e6-a3f3-4e00fe5c8f08"). InnerVolumeSpecName "kube-api-access-2bp96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:53:41.465913 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.465816    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "786d1b56-b737-47e6-a3f3-4e00fe5c8f08" (UID: "786d1b56-b737-47e6-a3f3-4e00fe5c8f08"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:53:41.465913 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.465836    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "786d1b56-b737-47e6-a3f3-4e00fe5c8f08" (UID: "786d1b56-b737-47e6-a3f3-4e00fe5c8f08"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:53:41.565537 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.565458    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:53:41.565537 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.565490    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:53:41.565537 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.565505    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bp96\" (UniqueName: \"kubernetes.io/projected/786d1b56-b737-47e6-a3f3-4e00fe5c8f08-kube-api-access-2bp96\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:53:41.653966 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:41.653937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:42.061382 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:42.061346    2577 generic.go:356] "Generic (PLEG): container finished" podID="786d1b56-b737-47e6-a3f3-4e00fe5c8f08" containerID="36aafe3e657cef42c8f899bfa32f743351ccccbc99688dac2d69ca4fe4d98ad8" exitCode=0
Sep 07 10:53:42.061801 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:42.061410    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"
Sep 07 10:53:42.061801 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:42.061438    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t" event={"ID":"786d1b56-b737-47e6-a3f3-4e00fe5c8f08","Type":"ContainerDied","Data":"36aafe3e657cef42c8f899bfa32f743351ccccbc99688dac2d69ca4fe4d98ad8"}
Sep 07 10:53:42.061801 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:42.061493    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t" event={"ID":"786d1b56-b737-47e6-a3f3-4e00fe5c8f08","Type":"ContainerDied","Data":"a9553d2fd5ec2c9ffa5d4ea0a038c4589cdebb44a70eedd004fe5f49f0964497"}
Sep 07 10:53:42.061801 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:42.061516    2577 scope.go:117] "RemoveContainer" containerID="36aafe3e657cef42c8f899bfa32f743351ccccbc99688dac2d69ca4fe4d98ad8"
Sep 07 10:53:42.074809 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:42.074787    2577 scope.go:117] "RemoveContainer" containerID="36aafe3e657cef42c8f899bfa32f743351ccccbc99688dac2d69ca4fe4d98ad8"
Sep 07 10:53:42.075163 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:53:42.075120    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36aafe3e657cef42c8f899bfa32f743351ccccbc99688dac2d69ca4fe4d98ad8\": container with ID starting with 36aafe3e657cef42c8f899bfa32f743351ccccbc99688dac2d69ca4fe4d98ad8 not found: ID does not exist" containerID="36aafe3e657cef42c8f899bfa32f743351ccccbc99688dac2d69ca4fe4d98ad8"
Sep 07 10:53:42.075247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:42.075174    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36aafe3e657cef42c8f899bfa32f743351ccccbc99688dac2d69ca4fe4d98ad8"} err="failed to get container status \"36aafe3e657cef42c8f899bfa32f743351ccccbc99688dac2d69ca4fe4d98ad8\": rpc error: code = NotFound desc = could not find container \"36aafe3e657cef42c8f899bfa32f743351ccccbc99688dac2d69ca4fe4d98ad8\": container with ID starting with 36aafe3e657cef42c8f899bfa32f743351ccccbc99688dac2d69ca4fe4d98ad8 not found: ID does not exist"
Sep 07 10:53:42.089269 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:42.089245    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"]
Sep 07 10:53:42.096603 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:42.096580    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-39c953-546588b47-wlg9t"]
Sep 07 10:53:42.410972 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:42.410886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:42.785450 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:42.785416    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="786d1b56-b737-47e6-a3f3-4e00fe5c8f08" path="/var/lib/kubelet/pods/786d1b56-b737-47e6-a3f3-4e00fe5c8f08/volumes"
Sep 07 10:53:46.731215 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:46.731181    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:53:47.934153 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:47.934073    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:48.034558 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:48.034520    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:48.223902 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:48.223854    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:52.316605 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:52.316570    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:53:57.117951 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:53:57.117918    2577 scope.go:117] "RemoveContainer" containerID="46d0b363dfcd172a3a94c0482286e3e1bbff80b3d60387c360cb529543c64395"
Sep 07 10:54:07.786990 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:07.786953    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"]
Sep 07 10:54:07.787454 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:07.787181    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv" podUID="f631bdfa-6ee7-442a-b736-40ae95ce0605" containerName="maas-api" containerID="cri-o://77f9ffb7ec79d234bde270c453cf4eee43afe0268c800773e0be5673c09aa0ee" gracePeriod=30
Sep 07 10:54:07.812681 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:07.812651    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:54:07.884273 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:07.884206    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:54:07.985980 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:07.985947    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:54:08.046813 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.046757    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:54:08.052178 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.052155    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:54:08.108743 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.108719    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f631bdfa-6ee7-442a-b736-40ae95ce0605-maas-api-tls\") pod \"f631bdfa-6ee7-442a-b736-40ae95ce0605\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") "
Sep 07 10:54:08.108882 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.108792    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f631bdfa-6ee7-442a-b736-40ae95ce0605-metrics-certs\") pod \"f631bdfa-6ee7-442a-b736-40ae95ce0605\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") "
Sep 07 10:54:08.108882 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.108843    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49dbl\" (UniqueName: \"kubernetes.io/projected/f631bdfa-6ee7-442a-b736-40ae95ce0605-kube-api-access-49dbl\") pod \"f631bdfa-6ee7-442a-b736-40ae95ce0605\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") "
Sep 07 10:54:08.108989 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.108952    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f631bdfa-6ee7-442a-b736-40ae95ce0605-openshift-service-ca\") pod \"f631bdfa-6ee7-442a-b736-40ae95ce0605\" (UID: \"f631bdfa-6ee7-442a-b736-40ae95ce0605\") "
Sep 07 10:54:08.109427 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.109395    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f631bdfa-6ee7-442a-b736-40ae95ce0605-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f631bdfa-6ee7-442a-b736-40ae95ce0605" (UID: "f631bdfa-6ee7-442a-b736-40ae95ce0605"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:54:08.111145 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.111100    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f631bdfa-6ee7-442a-b736-40ae95ce0605-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f631bdfa-6ee7-442a-b736-40ae95ce0605" (UID: "f631bdfa-6ee7-442a-b736-40ae95ce0605"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:54:08.111252 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.111116    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f631bdfa-6ee7-442a-b736-40ae95ce0605-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f631bdfa-6ee7-442a-b736-40ae95ce0605" (UID: "f631bdfa-6ee7-442a-b736-40ae95ce0605"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:54:08.111588 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.111566    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f631bdfa-6ee7-442a-b736-40ae95ce0605-kube-api-access-49dbl" (OuterVolumeSpecName: "kube-api-access-49dbl") pod "f631bdfa-6ee7-442a-b736-40ae95ce0605" (UID: "f631bdfa-6ee7-442a-b736-40ae95ce0605"). InnerVolumeSpecName "kube-api-access-49dbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:54:08.210563 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.210532    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f631bdfa-6ee7-442a-b736-40ae95ce0605-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:54:08.210563 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.210558    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f631bdfa-6ee7-442a-b736-40ae95ce0605-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:54:08.210563 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.210567    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49dbl\" (UniqueName: \"kubernetes.io/projected/f631bdfa-6ee7-442a-b736-40ae95ce0605-kube-api-access-49dbl\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:54:08.210793 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.210576    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f631bdfa-6ee7-442a-b736-40ae95ce0605-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:54:08.227907 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.227875    2577 generic.go:356] "Generic (PLEG): container finished" podID="f631bdfa-6ee7-442a-b736-40ae95ce0605" containerID="77f9ffb7ec79d234bde270c453cf4eee43afe0268c800773e0be5673c09aa0ee" exitCode=0
Sep 07 10:54:08.228033 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.227923    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"
Sep 07 10:54:08.228033 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.227961    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv" event={"ID":"f631bdfa-6ee7-442a-b736-40ae95ce0605","Type":"ContainerDied","Data":"77f9ffb7ec79d234bde270c453cf4eee43afe0268c800773e0be5673c09aa0ee"}
Sep 07 10:54:08.228033 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.228023    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv" event={"ID":"f631bdfa-6ee7-442a-b736-40ae95ce0605","Type":"ContainerDied","Data":"ee5ccbee1cef1ae7ccafea807dd889c2df695f6a8dafaaff9abdd25b1395d7ea"}
Sep 07 10:54:08.228194 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.228045    2577 scope.go:117] "RemoveContainer" containerID="77f9ffb7ec79d234bde270c453cf4eee43afe0268c800773e0be5673c09aa0ee"
Sep 07 10:54:08.238231 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.238212    2577 scope.go:117] "RemoveContainer" containerID="77f9ffb7ec79d234bde270c453cf4eee43afe0268c800773e0be5673c09aa0ee"
Sep 07 10:54:08.238604 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:54:08.238574    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77f9ffb7ec79d234bde270c453cf4eee43afe0268c800773e0be5673c09aa0ee\": container with ID starting with 77f9ffb7ec79d234bde270c453cf4eee43afe0268c800773e0be5673c09aa0ee not found: ID does not exist" containerID="77f9ffb7ec79d234bde270c453cf4eee43afe0268c800773e0be5673c09aa0ee"
Sep 07 10:54:08.238693 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.238615    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77f9ffb7ec79d234bde270c453cf4eee43afe0268c800773e0be5673c09aa0ee"} err="failed to get container status \"77f9ffb7ec79d234bde270c453cf4eee43afe0268c800773e0be5673c09aa0ee\": rpc error: code = NotFound desc = could not find container \"77f9ffb7ec79d234bde270c453cf4eee43afe0268c800773e0be5673c09aa0ee\": container with ID starting with 77f9ffb7ec79d234bde270c453cf4eee43afe0268c800773e0be5673c09aa0ee not found: ID does not exist"
Sep 07 10:54:08.248842 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.248820    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"]
Sep 07 10:54:08.253095 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.253074    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-bef02f-5b69b44f45-jgfxv"]
Sep 07 10:54:08.385254 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.385164    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:54:08.786971 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.786941    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f631bdfa-6ee7-442a-b736-40ae95ce0605" path="/var/lib/kubelet/pods/f631bdfa-6ee7-442a-b736-40ae95ce0605/volumes"
Sep 07 10:54:08.840648 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:08.840615    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:54:11.476245 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:11.476209    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:54:13.455185 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:13.455123    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:54:18.638722 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:18.638638    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:54:50.040267 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:50.040230    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:54:50.541291 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:50.541251    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:54:56.318418 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:56.318380    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:54:56.420454 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:56.420416    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:54:56.640623 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:56.640537    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:54:56.936877 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:56.936795    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:54:56.937414 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:56.937394    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:54:57.192453 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:54:57.192368    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:07.500804 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.500766    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"]
Sep 07 10:55:07.501203 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.500964    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf" podUID="a539b55e-2b82-488f-8123-956b2b85e91a" containerName="maas-api" containerID="cri-o://46f2f613ece19b236512f773a8833f7d60bac36ecf960e328f2bde55875c04a4" gracePeriod=30
Sep 07 10:55:07.523580 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.523540    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:07.623980 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.623945    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:07.724684 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.724648    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:07.751994 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.751942    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:55:07.827781 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.827745    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:07.845662 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.845636    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a539b55e-2b82-488f-8123-956b2b85e91a-maas-api-tls\") pod \"a539b55e-2b82-488f-8123-956b2b85e91a\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") "
Sep 07 10:55:07.845812 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.845678    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a539b55e-2b82-488f-8123-956b2b85e91a-openshift-service-ca\") pod \"a539b55e-2b82-488f-8123-956b2b85e91a\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") "
Sep 07 10:55:07.845812 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.845725    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a539b55e-2b82-488f-8123-956b2b85e91a-metrics-certs\") pod \"a539b55e-2b82-488f-8123-956b2b85e91a\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") "
Sep 07 10:55:07.845812 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.845796    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48rd5\" (UniqueName: \"kubernetes.io/projected/a539b55e-2b82-488f-8123-956b2b85e91a-kube-api-access-48rd5\") pod \"a539b55e-2b82-488f-8123-956b2b85e91a\" (UID: \"a539b55e-2b82-488f-8123-956b2b85e91a\") "
Sep 07 10:55:07.846146 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.846097    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a539b55e-2b82-488f-8123-956b2b85e91a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a539b55e-2b82-488f-8123-956b2b85e91a" (UID: "a539b55e-2b82-488f-8123-956b2b85e91a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:55:07.848041 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.848015    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a539b55e-2b82-488f-8123-956b2b85e91a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a539b55e-2b82-488f-8123-956b2b85e91a" (UID: "a539b55e-2b82-488f-8123-956b2b85e91a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:55:07.848401 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.848372    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a539b55e-2b82-488f-8123-956b2b85e91a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a539b55e-2b82-488f-8123-956b2b85e91a" (UID: "a539b55e-2b82-488f-8123-956b2b85e91a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:55:07.848549 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.848523    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a539b55e-2b82-488f-8123-956b2b85e91a-kube-api-access-48rd5" (OuterVolumeSpecName: "kube-api-access-48rd5") pod "a539b55e-2b82-488f-8123-956b2b85e91a" (UID: "a539b55e-2b82-488f-8123-956b2b85e91a"). InnerVolumeSpecName "kube-api-access-48rd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:55:07.947315 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.947275    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48rd5\" (UniqueName: \"kubernetes.io/projected/a539b55e-2b82-488f-8123-956b2b85e91a-kube-api-access-48rd5\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:55:07.947315 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.947307    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a539b55e-2b82-488f-8123-956b2b85e91a-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:55:07.947315 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.947317    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a539b55e-2b82-488f-8123-956b2b85e91a-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:55:07.947576 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:07.947325    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a539b55e-2b82-488f-8123-956b2b85e91a-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:55:08.183482 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:08.183388    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:08.593659 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:08.593622    2577 generic.go:356] "Generic (PLEG): container finished" podID="a539b55e-2b82-488f-8123-956b2b85e91a" containerID="46f2f613ece19b236512f773a8833f7d60bac36ecf960e328f2bde55875c04a4" exitCode=0
Sep 07 10:55:08.594108 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:08.593678    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"
Sep 07 10:55:08.594108 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:08.593683    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf" event={"ID":"a539b55e-2b82-488f-8123-956b2b85e91a","Type":"ContainerDied","Data":"46f2f613ece19b236512f773a8833f7d60bac36ecf960e328f2bde55875c04a4"}
Sep 07 10:55:08.594108 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:08.593769    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf" event={"ID":"a539b55e-2b82-488f-8123-956b2b85e91a","Type":"ContainerDied","Data":"b8d471ad29da42b57e218e6d289d807b05d190ca5a6979c970b01f626d1f2097"}
Sep 07 10:55:08.594108 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:08.593806    2577 scope.go:117] "RemoveContainer" containerID="46f2f613ece19b236512f773a8833f7d60bac36ecf960e328f2bde55875c04a4"
Sep 07 10:55:08.604093 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:08.604067    2577 scope.go:117] "RemoveContainer" containerID="46f2f613ece19b236512f773a8833f7d60bac36ecf960e328f2bde55875c04a4"
Sep 07 10:55:08.604419 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:55:08.604399    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46f2f613ece19b236512f773a8833f7d60bac36ecf960e328f2bde55875c04a4\": container with ID starting with 46f2f613ece19b236512f773a8833f7d60bac36ecf960e328f2bde55875c04a4 not found: ID does not exist" containerID="46f2f613ece19b236512f773a8833f7d60bac36ecf960e328f2bde55875c04a4"
Sep 07 10:55:08.604476 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:08.604428    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46f2f613ece19b236512f773a8833f7d60bac36ecf960e328f2bde55875c04a4"} err="failed to get container status \"46f2f613ece19b236512f773a8833f7d60bac36ecf960e328f2bde55875c04a4\": rpc error: code = NotFound desc = could not find container \"46f2f613ece19b236512f773a8833f7d60bac36ecf960e328f2bde55875c04a4\": container with ID starting with 46f2f613ece19b236512f773a8833f7d60bac36ecf960e328f2bde55875c04a4 not found: ID does not exist"
Sep 07 10:55:08.619072 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:08.619038    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"]
Sep 07 10:55:08.622430 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:08.622405    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-738923-797b85b747-zplbf"]
Sep 07 10:55:08.786590 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:08.786557    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a539b55e-2b82-488f-8123-956b2b85e91a" path="/var/lib/kubelet/pods/a539b55e-2b82-488f-8123-956b2b85e91a/volumes"
Sep 07 10:55:12.131718 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:12.131685    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:55:14.072952 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:14.072903    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:14.147213 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:14.147174    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:14.378543 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:14.378452    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:14.911821 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:14.911775    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:16.229967 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:16.229932    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:25.275509 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.275475    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"]
Sep 07 10:55:25.276088 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.275836    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz" podUID="71510b2c-caf0-49c3-879e-8fdc46cdd211" containerName="maas-api" containerID="cri-o://11d2e81292aa16cfe7e63cd10d518143e747f140b05cbe5a4775aa781b9a7b6d" gracePeriod=30
Sep 07 10:55:25.305723 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.305679    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:25.375459 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.375356    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:25.476704 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.476601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:25.526898 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.526838    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:55:25.549292 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.549260    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:25.606734 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.606702    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpfhf\" (UniqueName: \"kubernetes.io/projected/71510b2c-caf0-49c3-879e-8fdc46cdd211-kube-api-access-qpfhf\") pod \"71510b2c-caf0-49c3-879e-8fdc46cdd211\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") "
Sep 07 10:55:25.606940 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.606760    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71510b2c-caf0-49c3-879e-8fdc46cdd211-maas-api-tls\") pod \"71510b2c-caf0-49c3-879e-8fdc46cdd211\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") "
Sep 07 10:55:25.606940 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.606802    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71510b2c-caf0-49c3-879e-8fdc46cdd211-openshift-service-ca\") pod \"71510b2c-caf0-49c3-879e-8fdc46cdd211\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") "
Sep 07 10:55:25.606940 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.606823    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71510b2c-caf0-49c3-879e-8fdc46cdd211-metrics-certs\") pod \"71510b2c-caf0-49c3-879e-8fdc46cdd211\" (UID: \"71510b2c-caf0-49c3-879e-8fdc46cdd211\") "
Sep 07 10:55:25.607243 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.607215    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71510b2c-caf0-49c3-879e-8fdc46cdd211-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "71510b2c-caf0-49c3-879e-8fdc46cdd211" (UID: "71510b2c-caf0-49c3-879e-8fdc46cdd211"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:55:25.609212 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.609178    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71510b2c-caf0-49c3-879e-8fdc46cdd211-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "71510b2c-caf0-49c3-879e-8fdc46cdd211" (UID: "71510b2c-caf0-49c3-879e-8fdc46cdd211"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:55:25.609329 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.609271    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71510b2c-caf0-49c3-879e-8fdc46cdd211-kube-api-access-qpfhf" (OuterVolumeSpecName: "kube-api-access-qpfhf") pod "71510b2c-caf0-49c3-879e-8fdc46cdd211" (UID: "71510b2c-caf0-49c3-879e-8fdc46cdd211"). InnerVolumeSpecName "kube-api-access-qpfhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:55:25.609329 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.609270    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71510b2c-caf0-49c3-879e-8fdc46cdd211-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "71510b2c-caf0-49c3-879e-8fdc46cdd211" (UID: "71510b2c-caf0-49c3-879e-8fdc46cdd211"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:55:25.695182 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.695119    2577 generic.go:356] "Generic (PLEG): container finished" podID="71510b2c-caf0-49c3-879e-8fdc46cdd211" containerID="11d2e81292aa16cfe7e63cd10d518143e747f140b05cbe5a4775aa781b9a7b6d" exitCode=0
Sep 07 10:55:25.695182 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.695162    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz" event={"ID":"71510b2c-caf0-49c3-879e-8fdc46cdd211","Type":"ContainerDied","Data":"11d2e81292aa16cfe7e63cd10d518143e747f140b05cbe5a4775aa781b9a7b6d"}
Sep 07 10:55:25.695394 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.695192    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"
Sep 07 10:55:25.695394 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.695225    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz" event={"ID":"71510b2c-caf0-49c3-879e-8fdc46cdd211","Type":"ContainerDied","Data":"59c5916eb994f5e9358aa5c88f943d492dc45205f0f8b832403bb55fa7137caa"}
Sep 07 10:55:25.695394 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.695251    2577 scope.go:117] "RemoveContainer" containerID="11d2e81292aa16cfe7e63cd10d518143e747f140b05cbe5a4775aa781b9a7b6d"
Sep 07 10:55:25.705746 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.705729    2577 scope.go:117] "RemoveContainer" containerID="11d2e81292aa16cfe7e63cd10d518143e747f140b05cbe5a4775aa781b9a7b6d"
Sep 07 10:55:25.706044 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:55:25.706025    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11d2e81292aa16cfe7e63cd10d518143e747f140b05cbe5a4775aa781b9a7b6d\": container with ID starting with 11d2e81292aa16cfe7e63cd10d518143e747f140b05cbe5a4775aa781b9a7b6d not found: ID does not exist" containerID="11d2e81292aa16cfe7e63cd10d518143e747f140b05cbe5a4775aa781b9a7b6d"
Sep 07 10:55:25.706101 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.706054    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11d2e81292aa16cfe7e63cd10d518143e747f140b05cbe5a4775aa781b9a7b6d"} err="failed to get container status \"11d2e81292aa16cfe7e63cd10d518143e747f140b05cbe5a4775aa781b9a7b6d\": rpc error: code = NotFound desc = could not find container \"11d2e81292aa16cfe7e63cd10d518143e747f140b05cbe5a4775aa781b9a7b6d\": container with ID starting with 11d2e81292aa16cfe7e63cd10d518143e747f140b05cbe5a4775aa781b9a7b6d not found: ID does not exist"
Sep 07 10:55:25.707397 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.707378    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71510b2c-caf0-49c3-879e-8fdc46cdd211-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:55:25.707486 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.707399    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71510b2c-caf0-49c3-879e-8fdc46cdd211-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:55:25.707486 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.707410    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71510b2c-caf0-49c3-879e-8fdc46cdd211-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:55:25.707486 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.707420    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpfhf\" (UniqueName: \"kubernetes.io/projected/71510b2c-caf0-49c3-879e-8fdc46cdd211-kube-api-access-qpfhf\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:55:25.717102 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.717079    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"]
Sep 07 10:55:25.723190 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.723163    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-5a0ff8-5497f4bd67-r5jkz"]
Sep 07 10:55:25.939885 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:25.939777    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:26.791952 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:26.791918    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71510b2c-caf0-49c3-879e-8fdc46cdd211" path="/var/lib/kubelet/pods/71510b2c-caf0-49c3-879e-8fdc46cdd211/volumes"
Sep 07 10:55:30.131152 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:30.131097    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:55:31.955068 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:31.955031    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:32.027251 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:32.027190    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:32.272745 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:32.272660    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:32.923612 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:32.923578    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:43.191869 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.191826    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"]
Sep 07 10:55:43.192476 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.192090    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg" podUID="55b54502-7ac4-474c-ac1e-1ace55a068f4" containerName="maas-api" containerID="cri-o://ded16470125324f22277e9c24618b59e942d28d5adaf0b82406ee3ce4c8b1266" gracePeriod=30
Sep 07 10:55:43.222665 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.222633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:43.287561 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.287528    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:43.349956 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.349914    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:43.413455 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.413417    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:43.445466 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.445400    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:55:43.557402 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.557364    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55b54502-7ac4-474c-ac1e-1ace55a068f4-openshift-service-ca\") pod \"55b54502-7ac4-474c-ac1e-1ace55a068f4\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") "
Sep 07 10:55:43.557562 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.557411    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsvqr\" (UniqueName: \"kubernetes.io/projected/55b54502-7ac4-474c-ac1e-1ace55a068f4-kube-api-access-tsvqr\") pod \"55b54502-7ac4-474c-ac1e-1ace55a068f4\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") "
Sep 07 10:55:43.557562 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.557447    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55b54502-7ac4-474c-ac1e-1ace55a068f4-maas-api-tls\") pod \"55b54502-7ac4-474c-ac1e-1ace55a068f4\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") "
Sep 07 10:55:43.557562 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.557539    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55b54502-7ac4-474c-ac1e-1ace55a068f4-metrics-certs\") pod \"55b54502-7ac4-474c-ac1e-1ace55a068f4\" (UID: \"55b54502-7ac4-474c-ac1e-1ace55a068f4\") "
Sep 07 10:55:43.557798 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.557772    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55b54502-7ac4-474c-ac1e-1ace55a068f4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "55b54502-7ac4-474c-ac1e-1ace55a068f4" (UID: "55b54502-7ac4-474c-ac1e-1ace55a068f4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 10:55:43.559741 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.559704    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55b54502-7ac4-474c-ac1e-1ace55a068f4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "55b54502-7ac4-474c-ac1e-1ace55a068f4" (UID: "55b54502-7ac4-474c-ac1e-1ace55a068f4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:55:43.559854 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.559806    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55b54502-7ac4-474c-ac1e-1ace55a068f4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "55b54502-7ac4-474c-ac1e-1ace55a068f4" (UID: "55b54502-7ac4-474c-ac1e-1ace55a068f4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 10:55:43.559900 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.559849    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55b54502-7ac4-474c-ac1e-1ace55a068f4-kube-api-access-tsvqr" (OuterVolumeSpecName: "kube-api-access-tsvqr") pod "55b54502-7ac4-474c-ac1e-1ace55a068f4" (UID: "55b54502-7ac4-474c-ac1e-1ace55a068f4"). InnerVolumeSpecName "kube-api-access-tsvqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:55:43.658683 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.658649    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55b54502-7ac4-474c-ac1e-1ace55a068f4-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:55:43.658683 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.658678    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55b54502-7ac4-474c-ac1e-1ace55a068f4-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:55:43.658683 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.658688    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsvqr\" (UniqueName: \"kubernetes.io/projected/55b54502-7ac4-474c-ac1e-1ace55a068f4-kube-api-access-tsvqr\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:55:43.658888 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.658696    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55b54502-7ac4-474c-ac1e-1ace55a068f4-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:55:43.795261 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.795224    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:43.803913 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.803885    2577 generic.go:356] "Generic (PLEG): container finished" podID="55b54502-7ac4-474c-ac1e-1ace55a068f4" containerID="ded16470125324f22277e9c24618b59e942d28d5adaf0b82406ee3ce4c8b1266" exitCode=0
Sep 07 10:55:43.804083 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.803935    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"
Sep 07 10:55:43.804083 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.803969    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg" event={"ID":"55b54502-7ac4-474c-ac1e-1ace55a068f4","Type":"ContainerDied","Data":"ded16470125324f22277e9c24618b59e942d28d5adaf0b82406ee3ce4c8b1266"}
Sep 07 10:55:43.804083 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.804023    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg" event={"ID":"55b54502-7ac4-474c-ac1e-1ace55a068f4","Type":"ContainerDied","Data":"681f1f58230b7a1e9e547a7966329834c3a422bb46c256c314e6dcac85100b47"}
Sep 07 10:55:43.804083 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.804044    2577 scope.go:117] "RemoveContainer" containerID="ded16470125324f22277e9c24618b59e942d28d5adaf0b82406ee3ce4c8b1266"
Sep 07 10:55:43.813915 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.813898    2577 scope.go:117] "RemoveContainer" containerID="ded16470125324f22277e9c24618b59e942d28d5adaf0b82406ee3ce4c8b1266"
Sep 07 10:55:43.814214 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:55:43.814198    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ded16470125324f22277e9c24618b59e942d28d5adaf0b82406ee3ce4c8b1266\": container with ID starting with ded16470125324f22277e9c24618b59e942d28d5adaf0b82406ee3ce4c8b1266 not found: ID does not exist" containerID="ded16470125324f22277e9c24618b59e942d28d5adaf0b82406ee3ce4c8b1266"
Sep 07 10:55:43.814260 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.814223    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ded16470125324f22277e9c24618b59e942d28d5adaf0b82406ee3ce4c8b1266"} err="failed to get container status \"ded16470125324f22277e9c24618b59e942d28d5adaf0b82406ee3ce4c8b1266\": rpc error: code = NotFound desc = could not find container \"ded16470125324f22277e9c24618b59e942d28d5adaf0b82406ee3ce4c8b1266\": container with ID starting with ded16470125324f22277e9c24618b59e942d28d5adaf0b82406ee3ce4c8b1266 not found: ID does not exist"
Sep 07 10:55:43.833060 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.833030    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"]
Sep 07 10:55:43.837416 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:43.837397    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-659410-776dfd7777-xkmcg"]
Sep 07 10:55:44.792915 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:44.792812    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55b54502-7ac4-474c-ac1e-1ace55a068f4" path="/var/lib/kubelet/pods/55b54502-7ac4-474c-ac1e-1ace55a068f4/volumes"
Sep 07 10:55:45.931966 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:45.931930    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:55:49.707565 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:49.707472    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:49.826551 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:49.826505    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:49.939854 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:49.939822    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:55:55.332312 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:55.332273    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:55:57.158294 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:55:57.158259    2577 scope.go:117] "RemoveContainer" containerID="0c7737c7725cecbe57ce2e4f1a4cfffe5bb704816be3c3f4a708e60c5976f5b4"
Sep 07 10:57:16.264161 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:57:16.264112    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:57:17.233389 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:57:17.233352    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:57:18.143522 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:57:18.143489    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:57:19.686310 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:57:19.686269    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:57:43.929290 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:57:43.929253    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:57:50.837946 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:57:50.837908    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:57:58.333699 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:57:58.333663    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:58:05.826899 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:05.826856    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:58:18.343020 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:18.342981    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:58:26.728252 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:26.728217    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:58:34.154452 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.154415    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"]
Sep 07 10:58:34.154846 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.154626    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd" podUID="f3b4129a-d42c-4b66-b7c4-300dfb57d9d0" containerName="manager" containerID="cri-o://1bf14678aba8b43a760032008911c67e5e7cc4c097dd9cfa047ff065e379c47d" gracePeriod=10
Sep 07 10:58:34.517223 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.517197    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"
Sep 07 10:58:34.647410 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.647356    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj8q2\" (UniqueName: \"kubernetes.io/projected/f3b4129a-d42c-4b66-b7c4-300dfb57d9d0-kube-api-access-wj8q2\") pod \"f3b4129a-d42c-4b66-b7c4-300dfb57d9d0\" (UID: \"f3b4129a-d42c-4b66-b7c4-300dfb57d9d0\") "
Sep 07 10:58:34.647597 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.647483    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3b4129a-d42c-4b66-b7c4-300dfb57d9d0-extensions-socket-volume\") pod \"f3b4129a-d42c-4b66-b7c4-300dfb57d9d0\" (UID: \"f3b4129a-d42c-4b66-b7c4-300dfb57d9d0\") "
Sep 07 10:58:34.647888 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.647856    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3b4129a-d42c-4b66-b7c4-300dfb57d9d0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f3b4129a-d42c-4b66-b7c4-300dfb57d9d0" (UID: "f3b4129a-d42c-4b66-b7c4-300dfb57d9d0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 07 10:58:34.649654 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.649628    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3b4129a-d42c-4b66-b7c4-300dfb57d9d0-kube-api-access-wj8q2" (OuterVolumeSpecName: "kube-api-access-wj8q2") pod "f3b4129a-d42c-4b66-b7c4-300dfb57d9d0" (UID: "f3b4129a-d42c-4b66-b7c4-300dfb57d9d0"). InnerVolumeSpecName "kube-api-access-wj8q2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 10:58:34.748875 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.748840    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3b4129a-d42c-4b66-b7c4-300dfb57d9d0-extensions-socket-volume\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:58:34.748875 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.748871    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj8q2\" (UniqueName: \"kubernetes.io/projected/f3b4129a-d42c-4b66-b7c4-300dfb57d9d0-kube-api-access-wj8q2\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 10:58:34.785387 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.785358    2577 generic.go:356] "Generic (PLEG): container finished" podID="f3b4129a-d42c-4b66-b7c4-300dfb57d9d0" containerID="1bf14678aba8b43a760032008911c67e5e7cc4c097dd9cfa047ff065e379c47d" exitCode=0
Sep 07 10:58:34.785584 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.785407    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"
Sep 07 10:58:34.785584 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.785439    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd" event={"ID":"f3b4129a-d42c-4b66-b7c4-300dfb57d9d0","Type":"ContainerDied","Data":"1bf14678aba8b43a760032008911c67e5e7cc4c097dd9cfa047ff065e379c47d"}
Sep 07 10:58:34.785584 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.785497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd" event={"ID":"f3b4129a-d42c-4b66-b7c4-300dfb57d9d0","Type":"ContainerDied","Data":"8b14370fd639f8468405b12063911c265db523f89dc635751fb97925b9fe3e69"}
Sep 07 10:58:34.785584 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.785526    2577 scope.go:117] "RemoveContainer" containerID="1bf14678aba8b43a760032008911c67e5e7cc4c097dd9cfa047ff065e379c47d"
Sep 07 10:58:34.797019 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.796999    2577 scope.go:117] "RemoveContainer" containerID="1bf14678aba8b43a760032008911c67e5e7cc4c097dd9cfa047ff065e379c47d"
Sep 07 10:58:34.797367 ip-10-0-128-166 kubenswrapper[2577]: E0907 10:58:34.797345    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bf14678aba8b43a760032008911c67e5e7cc4c097dd9cfa047ff065e379c47d\": container with ID starting with 1bf14678aba8b43a760032008911c67e5e7cc4c097dd9cfa047ff065e379c47d not found: ID does not exist" containerID="1bf14678aba8b43a760032008911c67e5e7cc4c097dd9cfa047ff065e379c47d"
Sep 07 10:58:34.797440 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.797375    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bf14678aba8b43a760032008911c67e5e7cc4c097dd9cfa047ff065e379c47d"} err="failed to get container status \"1bf14678aba8b43a760032008911c67e5e7cc4c097dd9cfa047ff065e379c47d\": rpc error: code = NotFound desc = could not find container \"1bf14678aba8b43a760032008911c67e5e7cc4c097dd9cfa047ff065e379c47d\": container with ID starting with 1bf14678aba8b43a760032008911c67e5e7cc4c097dd9cfa047ff065e379c47d not found: ID does not exist"
Sep 07 10:58:34.812648 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.812615    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"]
Sep 07 10:58:34.815978 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:34.815957    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-t67xd"]
Sep 07 10:58:36.785106 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:36.785072    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3b4129a-d42c-4b66-b7c4-300dfb57d9d0" path="/var/lib/kubelet/pods/f3b4129a-d42c-4b66-b7c4-300dfb57d9d0/volumes"
Sep 07 10:58:42.267615 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:58:42.267576    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 10:59:34.881730 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.881692    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd"]
Sep 07 10:59:34.882247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882066    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3b4129a-d42c-4b66-b7c4-300dfb57d9d0" containerName="manager"
Sep 07 10:59:34.882247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882078    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3b4129a-d42c-4b66-b7c4-300dfb57d9d0" containerName="manager"
Sep 07 10:59:34.882247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882090    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="786d1b56-b737-47e6-a3f3-4e00fe5c8f08" containerName="maas-api"
Sep 07 10:59:34.882247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882097    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="786d1b56-b737-47e6-a3f3-4e00fe5c8f08" containerName="maas-api"
Sep 07 10:59:34.882247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882106    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55b54502-7ac4-474c-ac1e-1ace55a068f4" containerName="maas-api"
Sep 07 10:59:34.882247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882112    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="55b54502-7ac4-474c-ac1e-1ace55a068f4" containerName="maas-api"
Sep 07 10:59:34.882247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882121    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f631bdfa-6ee7-442a-b736-40ae95ce0605" containerName="maas-api"
Sep 07 10:59:34.882247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882126    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f631bdfa-6ee7-442a-b736-40ae95ce0605" containerName="maas-api"
Sep 07 10:59:34.882247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882154    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71510b2c-caf0-49c3-879e-8fdc46cdd211" containerName="maas-api"
Sep 07 10:59:34.882247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882159    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="71510b2c-caf0-49c3-879e-8fdc46cdd211" containerName="maas-api"
Sep 07 10:59:34.882247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882171    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a539b55e-2b82-488f-8123-956b2b85e91a" containerName="maas-api"
Sep 07 10:59:34.882247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882191    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a539b55e-2b82-488f-8123-956b2b85e91a" containerName="maas-api"
Sep 07 10:59:34.882247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882254    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f631bdfa-6ee7-442a-b736-40ae95ce0605" containerName="maas-api"
Sep 07 10:59:34.882247 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882261    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="55b54502-7ac4-474c-ac1e-1ace55a068f4" containerName="maas-api"
Sep 07 10:59:34.882688 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882269    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3b4129a-d42c-4b66-b7c4-300dfb57d9d0" containerName="manager"
Sep 07 10:59:34.882688 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882276    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="71510b2c-caf0-49c3-879e-8fdc46cdd211" containerName="maas-api"
Sep 07 10:59:34.882688 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882283    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a539b55e-2b82-488f-8123-956b2b85e91a" containerName="maas-api"
Sep 07 10:59:34.882688 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.882288    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="786d1b56-b737-47e6-a3f3-4e00fe5c8f08" containerName="maas-api"
Sep 07 10:59:34.885241 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.885221    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd"
Sep 07 10:59:34.887901 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.887883    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6h7k6\""
Sep 07 10:59:34.897505 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.897481    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd"]
Sep 07 10:59:34.973233 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.973192    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k4d4\" (UniqueName: \"kubernetes.io/projected/1523cbe2-2923-4301-8552-1abc5b8b0a4f-kube-api-access-6k4d4\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-zpmqd\" (UID: \"1523cbe2-2923-4301-8552-1abc5b8b0a4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd"
Sep 07 10:59:34.973410 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:34.973259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1523cbe2-2923-4301-8552-1abc5b8b0a4f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-zpmqd\" (UID: \"1523cbe2-2923-4301-8552-1abc5b8b0a4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd"
Sep 07 10:59:35.074498 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:35.074458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k4d4\" (UniqueName: \"kubernetes.io/projected/1523cbe2-2923-4301-8552-1abc5b8b0a4f-kube-api-access-6k4d4\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-zpmqd\" (UID: \"1523cbe2-2923-4301-8552-1abc5b8b0a4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd"
Sep 07 10:59:35.074677 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:35.074519    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1523cbe2-2923-4301-8552-1abc5b8b0a4f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-zpmqd\" (UID: \"1523cbe2-2923-4301-8552-1abc5b8b0a4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd"
Sep 07 10:59:35.074954 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:35.074934    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1523cbe2-2923-4301-8552-1abc5b8b0a4f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-zpmqd\" (UID: \"1523cbe2-2923-4301-8552-1abc5b8b0a4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd"
Sep 07 10:59:35.085872 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:35.085844    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k4d4\" (UniqueName: \"kubernetes.io/projected/1523cbe2-2923-4301-8552-1abc5b8b0a4f-kube-api-access-6k4d4\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-zpmqd\" (UID: \"1523cbe2-2923-4301-8552-1abc5b8b0a4f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd"
Sep 07 10:59:35.195339 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:35.195247    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd"
Sep 07 10:59:35.479095 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:35.479062    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd"]
Sep 07 10:59:35.481099 ip-10-0-128-166 kubenswrapper[2577]: W0907 10:59:35.481072    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1523cbe2_2923_4301_8552_1abc5b8b0a4f.slice/crio-a437d325ec4502ecdc03269be3aeaa395c9237fd8a119c00f6b96e3bec679673 WatchSource:0}: Error finding container a437d325ec4502ecdc03269be3aeaa395c9237fd8a119c00f6b96e3bec679673: Status 404 returned error can't find the container with id a437d325ec4502ecdc03269be3aeaa395c9237fd8a119c00f6b96e3bec679673
Sep 07 10:59:36.127704 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:36.127669    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd" event={"ID":"1523cbe2-2923-4301-8552-1abc5b8b0a4f","Type":"ContainerStarted","Data":"932d3bcf7dd7e50ee7b8e5a4a2f5b57cd285883e6fbbfc918da1b7b261331f50"}
Sep 07 10:59:36.128096 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:36.127722    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd" event={"ID":"1523cbe2-2923-4301-8552-1abc5b8b0a4f","Type":"ContainerStarted","Data":"a437d325ec4502ecdc03269be3aeaa395c9237fd8a119c00f6b96e3bec679673"}
Sep 07 10:59:36.128096 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:36.127776    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd"
Sep 07 10:59:36.155089 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:36.155037    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd" podStartSLOduration=2.155021622 podStartE2EDuration="2.155021622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 10:59:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 10:59:36.151465187 +0000 UTC m=+2379.885639423" watchObservedRunningTime="2026-09-07 10:59:36.155021622 +0000 UTC m=+2379.889195854"
Sep 07 10:59:47.136715 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:47.136679    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-zpmqd"
Sep 07 10:59:54.832691 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:54.832655    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 10:59:56.975953 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:56.975925    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 10:59:56.980612 ip-10-0-128-166 kubenswrapper[2577]: I0907 10:59:56.980584    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 11:00:02.939050 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:00:02.939011    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 11:00:06.352788 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:00:06.352751    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 11:00:07.047846 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:00:07.047812    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 11:00:10.834290 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:00:10.834250    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 11:00:21.239677 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:00:21.239589    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 11:00:24.740370 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:00:24.740333    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 11:01:07.792259 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:07.792227    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 11:01:09.495785 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:09.495755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 11:01:10.036915 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:10.036877    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 11:01:18.643653 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:18.643614    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 11:01:25.735769 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:25.735733    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 11:01:26.119812 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:26.119720    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 11:01:33.434710 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:33.434669    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 11:01:33.909020 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:33.908982    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"]
Sep 07 11:01:33.913147 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:33.913105    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:33.924950 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:33.924927    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"]
Sep 07 11:01:34.024338 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.024304    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75rjg\" (UniqueName: \"kubernetes.io/projected/5448afe1-0414-4287-9fbd-a4a8d6775247-kube-api-access-75rjg\") pod \"maas-api-84bbc9dfb6-thhln\" (UID: \"5448afe1-0414-4287-9fbd-a4a8d6775247\") " pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:34.024510 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.024360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5448afe1-0414-4287-9fbd-a4a8d6775247-openshift-service-ca\") pod \"maas-api-84bbc9dfb6-thhln\" (UID: \"5448afe1-0414-4287-9fbd-a4a8d6775247\") " pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:34.024510 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.024414    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5448afe1-0414-4287-9fbd-a4a8d6775247-maas-api-tls\") pod \"maas-api-84bbc9dfb6-thhln\" (UID: \"5448afe1-0414-4287-9fbd-a4a8d6775247\") " pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:34.024510 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.024440    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5448afe1-0414-4287-9fbd-a4a8d6775247-metrics-certs\") pod \"maas-api-84bbc9dfb6-thhln\" (UID: \"5448afe1-0414-4287-9fbd-a4a8d6775247\") " pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:34.125464 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.125424    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5448afe1-0414-4287-9fbd-a4a8d6775247-maas-api-tls\") pod \"maas-api-84bbc9dfb6-thhln\" (UID: \"5448afe1-0414-4287-9fbd-a4a8d6775247\") " pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:34.125464 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.125466    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5448afe1-0414-4287-9fbd-a4a8d6775247-metrics-certs\") pod \"maas-api-84bbc9dfb6-thhln\" (UID: \"5448afe1-0414-4287-9fbd-a4a8d6775247\") " pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:34.125697 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.125493    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75rjg\" (UniqueName: \"kubernetes.io/projected/5448afe1-0414-4287-9fbd-a4a8d6775247-kube-api-access-75rjg\") pod \"maas-api-84bbc9dfb6-thhln\" (UID: \"5448afe1-0414-4287-9fbd-a4a8d6775247\") " pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:34.125697 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.125546    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5448afe1-0414-4287-9fbd-a4a8d6775247-openshift-service-ca\") pod \"maas-api-84bbc9dfb6-thhln\" (UID: \"5448afe1-0414-4287-9fbd-a4a8d6775247\") " pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:34.126289 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.126211    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5448afe1-0414-4287-9fbd-a4a8d6775247-openshift-service-ca\") pod \"maas-api-84bbc9dfb6-thhln\" (UID: \"5448afe1-0414-4287-9fbd-a4a8d6775247\") " pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:34.128001 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.127980    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5448afe1-0414-4287-9fbd-a4a8d6775247-maas-api-tls\") pod \"maas-api-84bbc9dfb6-thhln\" (UID: \"5448afe1-0414-4287-9fbd-a4a8d6775247\") " pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:34.128213 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.128190    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5448afe1-0414-4287-9fbd-a4a8d6775247-metrics-certs\") pod \"maas-api-84bbc9dfb6-thhln\" (UID: \"5448afe1-0414-4287-9fbd-a4a8d6775247\") " pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:34.134229 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.134206    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75rjg\" (UniqueName: \"kubernetes.io/projected/5448afe1-0414-4287-9fbd-a4a8d6775247-kube-api-access-75rjg\") pod \"maas-api-84bbc9dfb6-thhln\" (UID: \"5448afe1-0414-4287-9fbd-a4a8d6775247\") " pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:34.225097 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.225061    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:34.499509 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.499483    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"]
Sep 07 11:01:34.501391 ip-10-0-128-166 kubenswrapper[2577]: W0907 11:01:34.501365    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5448afe1_0414_4287_9fbd_a4a8d6775247.slice/crio-1ad0d4ae33343a2b23916cd65afc6aec43f0b31151353c2882f5d9ac31215f92 WatchSource:0}: Error finding container 1ad0d4ae33343a2b23916cd65afc6aec43f0b31151353c2882f5d9ac31215f92: Status 404 returned error can't find the container with id 1ad0d4ae33343a2b23916cd65afc6aec43f0b31151353c2882f5d9ac31215f92
Sep 07 11:01:34.502954 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.502933    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 07 11:01:34.822185 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:34.822084    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln" event={"ID":"5448afe1-0414-4287-9fbd-a4a8d6775247","Type":"ContainerStarted","Data":"1ad0d4ae33343a2b23916cd65afc6aec43f0b31151353c2882f5d9ac31215f92"}
Sep 07 11:01:35.829673 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:35.829637    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln" event={"ID":"5448afe1-0414-4287-9fbd-a4a8d6775247","Type":"ContainerStarted","Data":"21356f756e01fd3e81f9e5d4450433f52186bf4b27383d1f7a84e69c53419e89"}
Sep 07 11:01:35.830166 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:35.829715    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:35.847070 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:35.847018    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln" podStartSLOduration=2.451453554 podStartE2EDuration="2.846996668s" totalImagesPullingTime="395.543114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:01:35.844783116 +0000 UTC m=+2499.578957362" watchObservedRunningTime="2026-09-07 11:01:35.846996668 +0000 UTC m=+2499.581170900"
Sep 07 11:01:41.840930 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:41.840901    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-84bbc9dfb6-thhln"
Sep 07 11:01:42.532893 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:42.532854    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 11:01:49.345293 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:49.345207    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"]
Sep 07 11:01:49.345827 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:49.345449    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s" podUID="469c398d-a67f-4a35-8e1e-32ae679ca5ca" containerName="maas-api" containerID="cri-o://8f868d5d4571defc3566504d15749e1af3e2ce005450a8596385516aadf48eb7" gracePeriod=30
Sep 07 11:01:49.417285 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:49.417247    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s" podUID="469c398d-a67f-4a35-8e1e-32ae679ca5ca" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.56:8443/health\": dial tcp 10.132.0.56:8443: connect: connection refused"
Sep 07 11:01:50.331985 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.331944    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 11:01:50.597579 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.597517    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 11:01:50.671608 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.671571    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/469c398d-a67f-4a35-8e1e-32ae679ca5ca-metrics-certs\") pod \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") "
Sep 07 11:01:50.671798 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.671623    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/469c398d-a67f-4a35-8e1e-32ae679ca5ca-openshift-service-ca\") pod \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") "
Sep 07 11:01:50.671798 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.671675    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/469c398d-a67f-4a35-8e1e-32ae679ca5ca-maas-api-tls\") pod \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") "
Sep 07 11:01:50.671798 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.671721    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjxzn\" (UniqueName: \"kubernetes.io/projected/469c398d-a67f-4a35-8e1e-32ae679ca5ca-kube-api-access-sjxzn\") pod \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\" (UID: \"469c398d-a67f-4a35-8e1e-32ae679ca5ca\") "
Sep 07 11:01:50.672092 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.672062    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/469c398d-a67f-4a35-8e1e-32ae679ca5ca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "469c398d-a67f-4a35-8e1e-32ae679ca5ca" (UID: "469c398d-a67f-4a35-8e1e-32ae679ca5ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 07 11:01:50.674065 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.674040    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/469c398d-a67f-4a35-8e1e-32ae679ca5ca-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "469c398d-a67f-4a35-8e1e-32ae679ca5ca" (UID: "469c398d-a67f-4a35-8e1e-32ae679ca5ca"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:01:50.674194 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.674109    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/469c398d-a67f-4a35-8e1e-32ae679ca5ca-kube-api-access-sjxzn" (OuterVolumeSpecName: "kube-api-access-sjxzn") pod "469c398d-a67f-4a35-8e1e-32ae679ca5ca" (UID: "469c398d-a67f-4a35-8e1e-32ae679ca5ca"). InnerVolumeSpecName "kube-api-access-sjxzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 07 11:01:50.674267 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.674217    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/469c398d-a67f-4a35-8e1e-32ae679ca5ca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "469c398d-a67f-4a35-8e1e-32ae679ca5ca" (UID: "469c398d-a67f-4a35-8e1e-32ae679ca5ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 07 11:01:50.772844 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.772809    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/469c398d-a67f-4a35-8e1e-32ae679ca5ca-metrics-certs\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 11:01:50.772844 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.772841    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/469c398d-a67f-4a35-8e1e-32ae679ca5ca-openshift-service-ca\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 11:01:50.772844 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.772851    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/469c398d-a67f-4a35-8e1e-32ae679ca5ca-maas-api-tls\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 11:01:50.773150 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.772860    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjxzn\" (UniqueName: \"kubernetes.io/projected/469c398d-a67f-4a35-8e1e-32ae679ca5ca-kube-api-access-sjxzn\") on node \"ip-10-0-128-166.ec2.internal\" DevicePath \"\""
Sep 07 11:01:50.914760 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.914667    2577 generic.go:356] "Generic (PLEG): container finished" podID="469c398d-a67f-4a35-8e1e-32ae679ca5ca" containerID="8f868d5d4571defc3566504d15749e1af3e2ce005450a8596385516aadf48eb7" exitCode=0
Sep 07 11:01:50.914760 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.914714    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"
Sep 07 11:01:50.914943 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.914720    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s" event={"ID":"469c398d-a67f-4a35-8e1e-32ae679ca5ca","Type":"ContainerDied","Data":"8f868d5d4571defc3566504d15749e1af3e2ce005450a8596385516aadf48eb7"}
Sep 07 11:01:50.914943 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.914834    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s" event={"ID":"469c398d-a67f-4a35-8e1e-32ae679ca5ca","Type":"ContainerDied","Data":"8ee11416a03abec30772f49f9e07eeda5ede3533f1f46df57e49dbf5697adf90"}
Sep 07 11:01:50.914943 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.914856    2577 scope.go:117] "RemoveContainer" containerID="8f868d5d4571defc3566504d15749e1af3e2ce005450a8596385516aadf48eb7"
Sep 07 11:01:50.924523 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.924502    2577 scope.go:117] "RemoveContainer" containerID="8f868d5d4571defc3566504d15749e1af3e2ce005450a8596385516aadf48eb7"
Sep 07 11:01:50.924797 ip-10-0-128-166 kubenswrapper[2577]: E0907 11:01:50.924778    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f868d5d4571defc3566504d15749e1af3e2ce005450a8596385516aadf48eb7\": container with ID starting with 8f868d5d4571defc3566504d15749e1af3e2ce005450a8596385516aadf48eb7 not found: ID does not exist" containerID="8f868d5d4571defc3566504d15749e1af3e2ce005450a8596385516aadf48eb7"
Sep 07 11:01:50.924849 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.924805    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f868d5d4571defc3566504d15749e1af3e2ce005450a8596385516aadf48eb7"} err="failed to get container status \"8f868d5d4571defc3566504d15749e1af3e2ce005450a8596385516aadf48eb7\": rpc error: code = NotFound desc = could not find container \"8f868d5d4571defc3566504d15749e1af3e2ce005450a8596385516aadf48eb7\": container with ID starting with 8f868d5d4571defc3566504d15749e1af3e2ce005450a8596385516aadf48eb7 not found: ID does not exist"
Sep 07 11:01:50.931722 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.931700    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"]
Sep 07 11:01:50.937291 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:50.937270    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-58ff47f946-pxd4s"]
Sep 07 11:01:52.784876 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:52.784843    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="469c398d-a67f-4a35-8e1e-32ae679ca5ca" path="/var/lib/kubelet/pods/469c398d-a67f-4a35-8e1e-32ae679ca5ca/volumes"
Sep 07 11:01:56.650172 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:01:56.650115    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 11:02:03.433264 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:03.433233    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-4qzwx"]
Sep 07 11:02:13.985231 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:13.985192    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gfk2q_462b889c-0701-47da-a0df-53ff3a1cc19d/manager/0.log"
Sep 07 11:02:14.127586 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:14.127557    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-sj2nj_43f23687-6876-4cb7-9b5f-856de753f721/manager/0.log"
Sep 07 11:02:14.892704 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:14.892675    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5bfcc886-76h68_118dfdf6-aae7-4c07-ac0b-7f0e42d93870/copy-manifests/0.log"
Sep 07 11:02:14.900189 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:14.900153    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5bfcc886-76h68_118dfdf6-aae7-4c07-ac0b-7f0e42d93870/manager/0.log"
Sep 07 11:02:15.585299 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:15.585274    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-84bbc9dfb6-thhln_5448afe1-0414-4287-9fbd-a4a8d6775247/maas-api/0.log"
Sep 07 11:02:17.076972 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:17.076904    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-cffps_6276af5a-6445-4829-9fdf-ab1262b939bf/manager/0.log"
Sep 07 11:02:17.212384 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:17.212348    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-dk5n6_9f7a4c3b-1dad-4023-8695-0539a0840895/manager/0.log"
Sep 07 11:02:17.346367 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:17.346291    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-hzrmh_311bf9fc-4c14-4211-a929-4e0142ca0df4/kuadrant-console-plugin/0.log"
Sep 07 11:02:17.730422 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:17.730390    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-zpmqd_1523cbe2-2923-4301-8552-1abc5b8b0a4f/manager/0.log"
Sep 07 11:02:17.907390 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:17.907354    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-4qzwx_ef69e096-39ae-40c6-8124-1e23cc1cd220/limitador/0.log"
Sep 07 11:02:18.052397 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:18.052319    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-vtn57_c0c4bdd8-1490-426c-b5c9-aad3aa2deb6a/manager/0.log"
Sep 07 11:02:18.291621 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:18.291582    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 11:02:18.848362 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:18.848331    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7794764676-62lq5_67fd58dd-c355-470a-8643-6dc31ef2ae9f/kube-auth-proxy/0.log"
Sep 07 11:02:19.573387 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:19.573353    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7676d8944-tzrbr_945b76dd-21f5-4690-b0fb-0afbf104601d/router/0.log"
Sep 07 11:02:20.310807 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:20.310766    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-ng874_d8425fa5-385c-4ef1-ba0c-ce39cf30de55/main/0.log"
Sep 07 11:02:20.574365 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:20.574273    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fd8829_426a17f8-51a8-49a8-8724-ea3f680dbd0c/main/0.log"
Sep 07 11:02:27.439317 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.439284    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jhxx4/must-gather-h5pz5"]
Sep 07 11:02:27.439771 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.439752    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="469c398d-a67f-4a35-8e1e-32ae679ca5ca" containerName="maas-api"
Sep 07 11:02:27.439857 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.439775    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="469c398d-a67f-4a35-8e1e-32ae679ca5ca" containerName="maas-api"
Sep 07 11:02:27.439923 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.439884    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="469c398d-a67f-4a35-8e1e-32ae679ca5ca" containerName="maas-api"
Sep 07 11:02:27.443591 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.443572    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jhxx4/must-gather-h5pz5"
Sep 07 11:02:27.446570 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.446548    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jhxx4\"/\"openshift-service-ca.crt\""
Sep 07 11:02:27.446679 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.446548    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jhxx4\"/\"default-dockercfg-jnqbs\""
Sep 07 11:02:27.447259 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.447242    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jhxx4\"/\"kube-root-ca.crt\""
Sep 07 11:02:27.451811 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.451779    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 11:02:27.453109 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.453081    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jhxx4/must-gather-h5pz5"]
Sep 07 11:02:27.545774 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.545742    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log" limit=1024
Sep 07 11:02:27.599591 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.599563    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6bb41684-1958-40b5-80e4-dd7ab8902139-must-gather-output\") pod \"must-gather-h5pz5\" (UID: \"6bb41684-1958-40b5-80e4-dd7ab8902139\") " pod="openshift-must-gather-jhxx4/must-gather-h5pz5"
Sep 07 11:02:27.599782 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.599627    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fr7q\" (UniqueName: \"kubernetes.io/projected/6bb41684-1958-40b5-80e4-dd7ab8902139-kube-api-access-2fr7q\") pod \"must-gather-h5pz5\" (UID: \"6bb41684-1958-40b5-80e4-dd7ab8902139\") " pod="openshift-must-gather-jhxx4/must-gather-h5pz5"
Sep 07 11:02:27.700751 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.700722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6bb41684-1958-40b5-80e4-dd7ab8902139-must-gather-output\") pod \"must-gather-h5pz5\" (UID: \"6bb41684-1958-40b5-80e4-dd7ab8902139\") " pod="openshift-must-gather-jhxx4/must-gather-h5pz5"
Sep 07 11:02:27.700918 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.700780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fr7q\" (UniqueName: \"kubernetes.io/projected/6bb41684-1958-40b5-80e4-dd7ab8902139-kube-api-access-2fr7q\") pod \"must-gather-h5pz5\" (UID: \"6bb41684-1958-40b5-80e4-dd7ab8902139\") " pod="openshift-must-gather-jhxx4/must-gather-h5pz5"
Sep 07 11:02:27.701143 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.701107    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6bb41684-1958-40b5-80e4-dd7ab8902139-must-gather-output\") pod \"must-gather-h5pz5\" (UID: \"6bb41684-1958-40b5-80e4-dd7ab8902139\") " pod="openshift-must-gather-jhxx4/must-gather-h5pz5"
Sep 07 11:02:27.710646 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.710623    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fr7q\" (UniqueName: \"kubernetes.io/projected/6bb41684-1958-40b5-80e4-dd7ab8902139-kube-api-access-2fr7q\") pod \"must-gather-h5pz5\" (UID: \"6bb41684-1958-40b5-80e4-dd7ab8902139\") " pod="openshift-must-gather-jhxx4/must-gather-h5pz5"
Sep 07 11:02:27.754351 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:27.754331    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jhxx4/must-gather-h5pz5"
Sep 07 11:02:28.027818 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:28.027795    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jhxx4/must-gather-h5pz5"]
Sep 07 11:02:28.030069 ip-10-0-128-166 kubenswrapper[2577]: W0907 11:02:28.030039    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bb41684_1958_40b5_80e4_dd7ab8902139.slice/crio-496f6aaa93e159edc9888eaff0cf190d2b2b9341a04c72806cb923eb6a872ef3 WatchSource:0}: Error finding container 496f6aaa93e159edc9888eaff0cf190d2b2b9341a04c72806cb923eb6a872ef3: Status 404 returned error can't find the container with id 496f6aaa93e159edc9888eaff0cf190d2b2b9341a04c72806cb923eb6a872ef3
Sep 07 11:02:28.133258 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:28.133228    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jhxx4/must-gather-h5pz5" event={"ID":"6bb41684-1958-40b5-80e4-dd7ab8902139","Type":"ContainerStarted","Data":"496f6aaa93e159edc9888eaff0cf190d2b2b9341a04c72806cb923eb6a872ef3"}
Sep 07 11:02:29.144414 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:29.144315    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jhxx4/must-gather-h5pz5" event={"ID":"6bb41684-1958-40b5-80e4-dd7ab8902139","Type":"ContainerStarted","Data":"f950394a93ae062fe85af434f501c9a8f63dca0638ecc01b50cb9ee0d9c8aa33"}
Sep 07 11:02:29.144414 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:29.144376    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jhxx4/must-gather-h5pz5" event={"ID":"6bb41684-1958-40b5-80e4-dd7ab8902139","Type":"ContainerStarted","Data":"d10d7020249196739ea239acff406b9bfe97b5b87537f3e259cb46b91c5d690b"}
Sep 07 11:02:29.162395 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:29.162332    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jhxx4/must-gather-h5pz5" podStartSLOduration=1.476054039 podStartE2EDuration="2.162311028s" totalImagesPullingTime="686.256989ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-07 11:02:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:02:29.160250198 +0000 UTC m=+2552.894424432" watchObservedRunningTime="2026-09-07 11:02:29.162311028 +0000 UTC m=+2552.896485262"
Sep 07 11:02:30.343878 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:30.343834    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ktvqp_804db42a-4910-4e6f-bdd2-d62cbb720877/global-pull-secret-syncer/0.log"
Sep 07 11:02:30.443472 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:30.443426    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7kpgf_f838808a-c1be-438f-b71b-b9148e6ec777/konnectivity-agent/0.log"
Sep 07 11:02:30.530040 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:30.530008    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-166.ec2.internal_91bd6355dd5f19af696dfdda483965bf/haproxy/0.log"
Sep 07 11:02:35.437540 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:35.437425    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-cffps_6276af5a-6445-4829-9fdf-ab1262b939bf/manager/0.log"
Sep 07 11:02:35.478572 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:35.478538    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-dk5n6_9f7a4c3b-1dad-4023-8695-0539a0840895/manager/0.log"
Sep 07 11:02:35.504623 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:35.504576    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-hzrmh_311bf9fc-4c14-4211-a929-4e0142ca0df4/kuadrant-console-plugin/0.log"
Sep 07 11:02:35.546546 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:35.546508    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-zpmqd_1523cbe2-2923-4301-8552-1abc5b8b0a4f/manager/0.log"
Sep 07 11:02:35.576894 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:35.576869    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-4qzwx_ef69e096-39ae-40c6-8124-1e23cc1cd220/limitador/0.log"
Sep 07 11:02:35.644835 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:35.644408    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-vtn57_c0c4bdd8-1490-426c-b5c9-aad3aa2deb6a/manager/0.log"
Sep 07 11:02:37.215864 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:37.215818    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-h87jj_b38067b2-aa31-4846-b669-8588a67c8fdf/cluster-monitoring-operator/0.log"
Sep 07 11:02:37.564153 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:37.563860    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hlqhq_4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0/node-exporter/0.log"
Sep 07 11:02:37.591817 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:37.591666    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hlqhq_4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0/kube-rbac-proxy/0.log"
Sep 07 11:02:37.624420 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:37.624381    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hlqhq_4fd1ba36-cbaa-4579-84f7-6bd47f81c4c0/init-textfile/0.log"
Sep 07 11:02:37.999247 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:37.999218    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-scc8r_c2407bec-ba3f-4c44-8908-c0d0b0e72316/prometheus-operator-admission-webhook/0.log"
Sep 07 11:02:39.631255 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:39.631222    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-djsc2_dc5e172d-81d8-4dd8-9ff7-26010c0ef971/networking-console-plugin/0.log"
Sep 07 11:02:40.060153 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.060082    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"]
Sep 07 11:02:40.066321 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.066287    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.072623 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.072587    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"]
Sep 07 11:02:40.127341 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.127303    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7ba78493-dfec-498f-a38d-8622e3ae6647-lib-modules\") pod \"perf-node-gather-daemonset-vltzs\" (UID: \"7ba78493-dfec-498f-a38d-8622e3ae6647\") " pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.127535 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.127385    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppvfr\" (UniqueName: \"kubernetes.io/projected/7ba78493-dfec-498f-a38d-8622e3ae6647-kube-api-access-ppvfr\") pod \"perf-node-gather-daemonset-vltzs\" (UID: \"7ba78493-dfec-498f-a38d-8622e3ae6647\") " pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.127535 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.127469    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ba78493-dfec-498f-a38d-8622e3ae6647-sys\") pod \"perf-node-gather-daemonset-vltzs\" (UID: \"7ba78493-dfec-498f-a38d-8622e3ae6647\") " pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.127535 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.127516    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7ba78493-dfec-498f-a38d-8622e3ae6647-podres\") pod \"perf-node-gather-daemonset-vltzs\" (UID: \"7ba78493-dfec-498f-a38d-8622e3ae6647\") " pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.127719 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.127582    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7ba78493-dfec-498f-a38d-8622e3ae6647-proc\") pod \"perf-node-gather-daemonset-vltzs\" (UID: \"7ba78493-dfec-498f-a38d-8622e3ae6647\") " pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.229080 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.229028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7ba78493-dfec-498f-a38d-8622e3ae6647-podres\") pod \"perf-node-gather-daemonset-vltzs\" (UID: \"7ba78493-dfec-498f-a38d-8622e3ae6647\") " pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.229306 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.229116    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7ba78493-dfec-498f-a38d-8622e3ae6647-proc\") pod \"perf-node-gather-daemonset-vltzs\" (UID: \"7ba78493-dfec-498f-a38d-8622e3ae6647\") " pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.229306 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.229194    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7ba78493-dfec-498f-a38d-8622e3ae6647-lib-modules\") pod \"perf-node-gather-daemonset-vltzs\" (UID: \"7ba78493-dfec-498f-a38d-8622e3ae6647\") " pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.229306 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.229242    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppvfr\" (UniqueName: \"kubernetes.io/projected/7ba78493-dfec-498f-a38d-8622e3ae6647-kube-api-access-ppvfr\") pod \"perf-node-gather-daemonset-vltzs\" (UID: \"7ba78493-dfec-498f-a38d-8622e3ae6647\") " pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.229492 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.229311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ba78493-dfec-498f-a38d-8622e3ae6647-sys\") pod \"perf-node-gather-daemonset-vltzs\" (UID: \"7ba78493-dfec-498f-a38d-8622e3ae6647\") " pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.229492 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.229412    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7ba78493-dfec-498f-a38d-8622e3ae6647-proc\") pod \"perf-node-gather-daemonset-vltzs\" (UID: \"7ba78493-dfec-498f-a38d-8622e3ae6647\") " pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.229594 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.229524    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7ba78493-dfec-498f-a38d-8622e3ae6647-podres\") pod \"perf-node-gather-daemonset-vltzs\" (UID: \"7ba78493-dfec-498f-a38d-8622e3ae6647\") " pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.229594 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.229533    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7ba78493-dfec-498f-a38d-8622e3ae6647-lib-modules\") pod \"perf-node-gather-daemonset-vltzs\" (UID: \"7ba78493-dfec-498f-a38d-8622e3ae6647\") " pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.229594 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.229548    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7ba78493-dfec-498f-a38d-8622e3ae6647-sys\") pod \"perf-node-gather-daemonset-vltzs\" (UID: \"7ba78493-dfec-498f-a38d-8622e3ae6647\") " pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.239046 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.239014    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppvfr\" (UniqueName: \"kubernetes.io/projected/7ba78493-dfec-498f-a38d-8622e3ae6647-kube-api-access-ppvfr\") pod \"perf-node-gather-daemonset-vltzs\" (UID: \"7ba78493-dfec-498f-a38d-8622e3ae6647\") " pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.287953 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.287913    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/1.log"
Sep 07 11:02:40.295231 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.295200    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-grp6m_72995f6c-69b6-468a-86e2-2471b01e2e4d/console-operator/2.log"
Sep 07 11:02:40.388395 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.388307    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:40.691263 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.690905    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"]
Sep 07 11:02:40.692892 ip-10-0-128-166 kubenswrapper[2577]: W0907 11:02:40.692864    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7ba78493_dfec_498f_a38d_8622e3ae6647.slice/crio-94754c86954d489fe8ab9eff101574b641a7fe7e70e9f900f9391511cb3a51b5 WatchSource:0}: Error finding container 94754c86954d489fe8ab9eff101574b641a7fe7e70e9f900f9391511cb3a51b5: Status 404 returned error can't find the container with id 94754c86954d489fe8ab9eff101574b641a7fe7e70e9f900f9391511cb3a51b5
Sep 07 11:02:40.855314 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.855212    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-56c94f66c8-4qf99_a5299f7c-f528-4b86-9f30-5e13df5f3c9a/console/0.log"
Sep 07 11:02:40.898165 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:40.898121    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-7rng4_c1de07df-6d06-4584-96e2-0bd0f5d5b418/download-server/0.log"
Sep 07 11:02:41.255229 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:41.255187    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs" event={"ID":"7ba78493-dfec-498f-a38d-8622e3ae6647","Type":"ContainerStarted","Data":"e592e5fd9ae6c4774ebbd9010de50592ffb9b0fd02a21568d4bd45ac262ba178"}
Sep 07 11:02:41.255439 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:41.255263    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs" event={"ID":"7ba78493-dfec-498f-a38d-8622e3ae6647","Type":"ContainerStarted","Data":"94754c86954d489fe8ab9eff101574b641a7fe7e70e9f900f9391511cb3a51b5"}
Sep 07 11:02:41.255913 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:41.255888    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:41.273788 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:41.273729    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs" podStartSLOduration=1.273709875 podStartE2EDuration="1.273709875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-07 11:02:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-07 11:02:41.27135849 +0000 UTC m=+2565.005532726" watchObservedRunningTime="2026-09-07 11:02:41.273709875 +0000 UTC m=+2565.007884108"
Sep 07 11:02:41.585553 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:41.585459    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-x5vgs_b2626e1c-b66e-4c2a-8d52-399b53bf16c8/volume-data-source-validator/0.log"
Sep 07 11:02:42.712622 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:42.712573    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5g6z4_ce8a104b-d12d-46bf-accb-e8ed7d04195c/dns/0.log"
Sep 07 11:02:42.734961 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:42.734929    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5g6z4_ce8a104b-d12d-46bf-accb-e8ed7d04195c/kube-rbac-proxy/0.log"
Sep 07 11:02:42.874827 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:42.874801    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4hg64_0187e151-799f-402c-9886-6cb9b0bba10d/dns-node-resolver/0.log"
Sep 07 11:02:43.496873 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:43.496839    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mljbz_bcc4fa7a-3956-4a04-8fc7-b1914d6079f8/node-ca/0.log"
Sep 07 11:02:44.774120 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:44.774055    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7794764676-62lq5_67fd58dd-c355-470a-8643-6dc31ef2ae9f/kube-auth-proxy/0.log"
Sep 07 11:02:44.930627 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:44.930595    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7676d8944-tzrbr_945b76dd-21f5-4690-b0fb-0afbf104601d/router/0.log"
Sep 07 11:02:45.830536 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:45.830503    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tc4rv_1fac04e7-2860-4151-88df-b67485cd4f19/serve-healthcheck-canary/0.log"
Sep 07 11:02:46.308455 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:46.308422    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-srlkt_228ba212-8e2a-4dab-8381-cc40eb711df7/insights-operator/0.log"
Sep 07 11:02:46.311882 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:46.311852    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-srlkt_228ba212-8e2a-4dab-8381-cc40eb711df7/insights-operator/1.log"
Sep 07 11:02:46.332852 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:46.332823    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p9h4t_1189ab74-04bb-4104-ae57-cc5fe35a3fbd/kube-rbac-proxy/0.log"
Sep 07 11:02:46.356720 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:46.356693    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p9h4t_1189ab74-04bb-4104-ae57-cc5fe35a3fbd/exporter/0.log"
Sep 07 11:02:46.379642 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:46.379609    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p9h4t_1189ab74-04bb-4104-ae57-cc5fe35a3fbd/extractor/0.log"
Sep 07 11:02:48.282249 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:48.282220    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jhxx4/perf-node-gather-daemonset-vltzs"
Sep 07 11:02:48.636638 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:48.636565    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gfk2q_462b889c-0701-47da-a0df-53ff3a1cc19d/manager/0.log"
Sep 07 11:02:48.979356 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:48.979319    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-sj2nj_43f23687-6876-4cb7-9b5f-856de753f721/manager/0.log"
Sep 07 11:02:49.239838 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:49.239753    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5bfcc886-76h68_118dfdf6-aae7-4c07-ac0b-7f0e42d93870/manager/0.log"
Sep 07 11:02:49.260228 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:49.260204    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5bfcc886-76h68_118dfdf6-aae7-4c07-ac0b-7f0e42d93870/copy-manifests/0.log"
Sep 07 11:02:51.086859 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:51.086769    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7794764676-62lq5_67fd58dd-c355-470a-8643-6dc31ef2ae9f/kube-auth-proxy/0.log"
Sep 07 11:02:51.213276 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:51.213241    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7676d8944-tzrbr_945b76dd-21f5-4690-b0fb-0afbf104601d/router/0.log"
Sep 07 11:02:53.021834 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:53.021766    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7794764676-62lq5_67fd58dd-c355-470a-8643-6dc31ef2ae9f/kube-auth-proxy/0.log"
Sep 07 11:02:53.141197 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:53.141098    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7676d8944-tzrbr_945b76dd-21f5-4690-b0fb-0afbf104601d/router/0.log"
Sep 07 11:02:55.859054 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:55.859022    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-lsg9w_7766f9d3-1516-4284-8a1b-fb443dc30fa8/migrator/0.log"
Sep 07 11:02:55.881372 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:55.881323    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-lsg9w_7766f9d3-1516-4284-8a1b-fb443dc30fa8/graceful-termination/0.log"
Sep 07 11:02:56.238567 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:56.238536    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-8rktn_1ad9111f-27a6-4f1f-b886-66c18049ccb6/kube-storage-version-migrator-operator/1.log"
Sep 07 11:02:56.240339 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:56.240302    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-8rktn_1ad9111f-27a6-4f1f-b886-66c18049ccb6/kube-storage-version-migrator-operator/0.log"
Sep 07 11:02:57.433397 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:57.433367    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hp27c_ac75d0ea-397b-457f-b8b1-46735f0dff26/kube-multus-additional-cni-plugins/0.log"
Sep 07 11:02:57.455281 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:57.455255    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hp27c_ac75d0ea-397b-457f-b8b1-46735f0dff26/egress-router-binary-copy/0.log"
Sep 07 11:02:57.480503 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:57.480473    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hp27c_ac75d0ea-397b-457f-b8b1-46735f0dff26/cni-plugins/0.log"
Sep 07 11:02:57.503831 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:57.503809    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hp27c_ac75d0ea-397b-457f-b8b1-46735f0dff26/bond-cni-plugin/0.log"
Sep 07 11:02:57.525324 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:57.525303    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hp27c_ac75d0ea-397b-457f-b8b1-46735f0dff26/routeoverride-cni/0.log"
Sep 07 11:02:57.546224 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:57.546177    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hp27c_ac75d0ea-397b-457f-b8b1-46735f0dff26/whereabouts-cni-bincopy/0.log"
Sep 07 11:02:57.568880 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:57.568856    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hp27c_ac75d0ea-397b-457f-b8b1-46735f0dff26/whereabouts-cni/0.log"
Sep 07 11:02:57.863329 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:57.863282    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pjkq5_0e183d73-394f-4794-a848-d217bb88c3fa/kube-multus/0.log"
Sep 07 11:02:57.890642 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:57.890614    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-x8w2s_415363e6-edcf-41e5-97ce-cd379cf30f4e/network-metrics-daemon/0.log"
Sep 07 11:02:57.911801 ip-10-0-128-166 kubenswrapper[2577]: I0907 11:02:57.911762    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-x8w2s_415363e6-edcf-41e5-97ce-cd379cf30f4e/kube-rbac-proxy/0.log"